Multi-participant live communication user interface

ABSTRACT

An electronic device is disclosed. The device can display a live video communication user interface of a live video communication application that includes participant affordances corresponding to participants. In response to detecting an input that corresponds to a request to initiate a process of starting a new live communication session, the device can: in accordance with the input corresponding to selection of the first participant affordance, initiate a process of starting a new live communication session that includes a group of three or more participants; and in accordance with the input corresponding to selection of the new session affordance, initiate a process for selecting two or more participants and start a new live communication session that includes a participant associated with the device and the selected participants.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/157,166, titled “MULTI-PARTICIPANT LIVE COMMUNICATION USERINTERFACE,” filed Jan. 25, 2021, which is a continuation of U.S. patentapplication Ser. No. 16/799,481, titled “MULTI-PARTICIPANT LIVECOMMUNICATION USER INTERFACE,” filed Feb. 24, 2020, which is acontinuation of U.S. patent application Ser. No. 16/383,403, titled“MULTI-PARTICIPANT LIVE COMMUNICATION USER INTERFACE,” filed Apr. 12,2019, which is a continuation of U.S. patent application Ser. No.16/144,572, titled “MULTI-PARTICIPANT LIVE COMMUNICATION USERINTERFACE,” filed on Sep. 27, 2018, which claims priority to U.S.Provisional Patent Application Ser. No. 62/679,925, titled“MULTI-PARTICIPANT LIVE COMMUNICATION USER INTERFACE,” filed on Jun. 3,2018; and U.S. Provisional Patent Application Ser. No. 62/668,229,titled “MULTI-PARTICIPANT LIVE COMMUNICATION USER INTERFACE,” filed onMay 7, 2018, the contents of each of which are hereby incorporated byreference in their entireties.

FIELD

The present disclosure relates generally to computer user interfaces,and more specifically to techniques for live communication involvingmultiple participants.

BACKGROUND

Users are increasingly using electronic devices to virtually communicatewith others for personal and business reasons. Most electronic devicesare capable of providing live communication between users and multipleparticipants. As an example, some electronic devices provide interactiveinterfaces to display and control the live communication.

BRIEF SUMMARY

Some techniques for live communication involving multiple participantsusing electronic devices, however, are generally cumbersome andinefficient. For example, some existing techniques use a complex andtime-consuming user interface, which may include multiple key presses orkeystrokes. Existing techniques require more time than necessary,wasting user time and device energy. This latter consideration isparticularly important in battery-operated devices.

Accordingly, the present technique provides electronic devices withfaster, more efficient methods and interfaces for live communicationinvolving multiple participants. Such methods and interfaces optionallycomplement or replace other methods for live communication involvingmultiple participants. Such methods and interfaces reduce the cognitiveburden on a user and produce a more efficient human-machine interface.For battery-operated computing devices, such methods and interfacesconserve power and increase the time between battery charges.

In some embodiments, a method is performed at an electronic device witha display. The method may include: displaying a messaging user interfaceof a messaging application, the messaging user interface including amessage region that includes a plurality of messages in a messageconversation between three or more participants; receiving first dataindicating that a live communication session that is available to thethree or more participants is active; in response to receiving the firstdata indicating that the live communication session between the three ormore participants is active, displaying a notification indicating thatthe live communication session is active; while displaying thenotification indicating that the live communication session is active,receiving second data indicating that the live communication sessionbetween the three or more participants is no longer active; and inresponse to receiving the second data indicating that the livecommunication session between the three or more participants is nolonger active, ceasing to display the notification or updating thenotification to indicate that the live communication session is nolonger active.

In some embodiments, a non-transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a messaging user interfaceof a messaging application, the messaging user interface including amessage region that includes a plurality of messages in a messageconversation between three or more participants; receiving first dataindicating that a live communication session that is available to thethree or more participants is active; in response to receiving the firstdata indicating that the live communication session between the three ormore participants is active, displaying a notification indicating thatthe live communication session is active; while displaying thenotification indicating that the live communication session is active,receiving second data indicating that the live communication sessionbetween the three or more participants is no longer active; and inresponse to receiving the second data indicating that the livecommunication session between the three or more participants is nolonger active, ceasing to display the notification or updating thenotification to indicate that the live communication session is nolonger active.

In some embodiments, a transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a messaging user interfaceof a messaging application, the messaging user interface including amessage region that includes a plurality of messages in a messageconversation between three or more participants; receiving first dataindicating that a live communication session that is available to thethree or more participants is active. In response to receiving the firstdata indicating that the live communication session between the three ormore participants is active, displaying a notification indicating thatthe live communication session is active; while displaying thenotification indicating that the live communication session is active,receiving second data indicating that the live communication sessionbetween the three or more participants is no longer active; and inresponse to receiving the second data indicating that the livecommunication session between the three or more participants is nolonger active, ceasing to display the notification or updating thenotification to indicate that the live communication session is nolonger active.

In some embodiments, an electronic device includes a display, one ormore processors, and memory. The memory stores one or more programsconfigured to be executed by the one or more processors. The one or moreprograms including instructions for: displaying a messaging userinterface of a messaging application, the messaging user interfaceincluding a message region that includes a plurality of messages in amessage conversation between three or more participants; receiving firstdata indicating that a live communication session that is available tothe three or more participants is active. In response to receiving thefirst data indicating that the live communication session between thethree or more participants is active, displaying a notificationindicating that the live communication session is active; whiledisplaying the notification indicating that the live communicationsession is active, receiving second data indicating that the livecommunication session between the three or more participants is nolonger active; and in response to receiving the second data indicatingthat the live communication session between the three or moreparticipants is no longer active, ceasing to display the notification orupdating the notification to indicate that the live communicationsession is no longer active.

In some embodiments, an electronic device includes: a display; means fordisplaying a messaging user interface of a messaging application, themessaging user interface including a message region that includes aplurality of messages in a message conversation between three or moreparticipants; means for receiving first data indicating that a livecommunication session that is available to the three or moreparticipants is active; means responsive to receiving the first dataindicating that the live communication session between the three or moreparticipants is active, displaying a notification indicating that thelive communication session is active; means for, while displaying thenotification indicating that the live communication session is active,receiving second data indicating that the live communication sessionbetween the three or more participants is no longer active; and meansfor responsive to receiving the second data indicating that the livecommunication session between the three or more participants is nolonger active, ceasing to display the notification or updating thenotification to indicate that the live communication session is nolonger active.

In some embodiments, a method is performed at an electronic device witha display. The method may include: receiving a request to display a userinterface for a live communication session between two or moreparticipants; and in response to receiving the request to display theuser interface for the live communication session between two or moreparticipants, displaying a live communication user interface includingconcurrently displaying a plurality of representations of participantsin the live communication session, where displaying the plurality ofrepresentations of participants in the live communication sessionincludes: in accordance with a determination that the number ofparticipants in the live communication session includes more than athreshold number of participants, concurrently displayingrepresentations of a first plurality of the participants other than auser of the electronic device at one or more sizes greater than arespective threshold size while displaying representations of a secondplurality of the participants in an overflow region, where the secondplurality of participants includes one or more participants not includedin the first plurality of participants and the representations of theone or more other participants displayed in the overflow region aredisplayed at a size smaller than the respective threshold size; and inaccordance with a determination that the number of participants in thelive communication session includes less than or equal to the thresholdnumber of participants, displaying representations of the participantsother than the user of the electronic device at one or more sizesgreater than a respective threshold size without displayingrepresentations of participants other than the user of the electronicdevice at one or more sizes smaller than or equal to the respectivethreshold size.

In some embodiments, a non-transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: receiving a request to display a userinterface for a live communication session between two or moreparticipants; and in response to receiving the request to display theuser interface for the live communication session between two or moreparticipants, displaying a live communication user interface includingconcurrently displaying a plurality of representations of participantsin the live communication session, where displaying the plurality ofrepresentations of participants in the live communication sessionincludes: in accordance with a determination that the number ofparticipants in the live communication session includes more than athreshold number of participants, concurrently displayingrepresentations of a first plurality of the participants other than auser of the electronic device at one or more sizes greater than arespective threshold size while displaying representations of a secondplurality of the participants in an overflow region, where the secondplurality of participants includes one or more participants not includedin the first plurality of participants and the representations of theone or more other participants displayed in the overflow region aredisplayed at a size smaller than the respective threshold size; and inaccordance with a determination that the number of participants in thelive communication session includes less than or equal to the thresholdnumber of participants, displaying representations of the participantsother than the user of the electronic device at one or more sizesgreater than a respective threshold size without displayingrepresentations of participants other than the user of the electronicdevice at one or more sizes smaller than or equal to the respectivethreshold size.

In some embodiments, a transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: receiving a request to display a userinterface for a live communication session between two or moreparticipants; and in response to receiving the request to display theuser interface for the live communication session between two or moreparticipants, displaying a live communication user interface includingconcurrently displaying a plurality of representations of participantsin the live communication session, where displaying the plurality ofrepresentations of participants in the live communication sessionincludes: in accordance with a determination that the number ofparticipants in the live communication session includes more than athreshold number of participants, concurrently displayingrepresentations of a first plurality of the participants other than auser of the electronic device at one or more sizes greater than arespective threshold size while displaying representations of a secondplurality of the participants in an overflow region, where the secondplurality of participants includes one or more participants not includedin the first plurality of participants and the representations of theone or more other participants displayed in the overflow region aredisplayed at a size smaller than the respective threshold size; and inaccordance with a determination that the number of participants in thelive communication session includes less than or equal to the thresholdnumber of participants, displaying representations of the participantsother than the user of the electronic device at one or more sizesgreater than a respective threshold size without displayingrepresentations of participants other than the user of the electronicdevice at one or more sizes smaller than or equal to the respectivethreshold size.

In some embodiments, an electronic device includes a display, one ormore processors, and memory. The memory stores one or more programsconfigured to be executed by the one or more processors. The one or moreprograms including instructions for: receiving a request to display auser interface for a live communication session between two or moreparticipants; and in response to receiving the request to display theuser interface for the live communication session between two or moreparticipants, displaying a live communication user interface includingconcurrently displaying a plurality of representations of participantsin the live communication session, where displaying the plurality ofrepresentations of participants in the live communication sessionincludes: in accordance with a determination that the number ofparticipants in the live communication session includes more than athreshold number of participants, concurrently displayingrepresentations of a first plurality of the participants other than auser of the electronic device at one or more sizes greater than arespective threshold size while displaying representations of a secondplurality of the participants in an overflow region, where the secondplurality of participants includes one or more participants not includedin the first plurality of participants and the representations of theone or more other participants displayed in the overflow region aredisplayed at a size smaller than the respective threshold size; and inaccordance with a determination that the number of participants in thelive communication session includes less than or equal to the thresholdnumber of participants, displaying representations of the participantsother than the user of the electronic device at one or more sizesgreater than a respective threshold size without displayingrepresentations of participants other than the user of the electronicdevice at one or more sizes smaller than or equal to the respectivethreshold size.

In some embodiments, an electronic device includes: a display; means forreceiving a request to display a user interface for a live communicationsession between two or more participants; and means responsive toreceiving the request to display the user interface for the livecommunication session between two or more participants, for displaying alive communication user interface including concurrently displaying aplurality of representations of participants in the live communicationsession, where displaying the plurality of representations ofparticipants in the live communication session includes: means, inaccordance with a determination that the number of participants in thelive communication session includes more than a threshold number ofparticipants, for concurrently displaying representations of a firstplurality of the participants other than a user of the electronic deviceat one or more sizes greater than a respective threshold size whiledisplaying representations of a second plurality of the participants inan overflow region, where the second plurality of participants includesone or more participants not included in the first plurality ofparticipants and the representations of the one or more otherparticipants displayed in the overflow region are displayed at a sizesmaller than the respective threshold size; and means, in accordancewith a determination that the number of participants in the livecommunication session includes less than or equal to the thresholdnumber of participants, for displaying representations of theparticipants other than the user of the electronic device at one or moresizes greater than a respective threshold size without displayingrepresentations of participants other than the user of the electronicdevice at one or more sizes smaller than or equal to the respectivethreshold size.

In some embodiments, a method is performed at an electronic device witha display. The method may include: displaying a live video communicationuser interface that includes concurrently displaying a representation ofa first participant in a live video communication session, arepresentation of a second participant in the live video communicationsession, and a representation of a third participant in the live videocommunication session; while displaying the live video communicationuser interface, receiving data indicating that an activity level of thesecond participant in the live video communication session has increasedabove the activity level of the first participant in the live videocommunication session; and in response to receiving the data indicatingthat the activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session: changing asize and/or position of the representation of the first participant inthe live video communication user interface; changing a size and/orposition of the representation of the second participant in the livevideo communication user interface; and changing a size and/or positionof the representation of the third participant in the live videocommunication user interface.

In some embodiments, a non-transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a live video communicationuser interface that includes concurrently displaying a representation ofa first participant in a live video communication session, arepresentation of a second participant in the live video communicationsession, and a representation of a third participant in the live videocommunication session; while displaying the live video communicationuser interface, receiving data indicating that an activity level of thesecond participant in the live video communication session has increasedabove the activity level of the first participant in the live videocommunication session; and in response to receiving the data indicatingthat the activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session: changing asize and/or position of the representation of the first participant inthe live video communication user interface; changing a size and/orposition of the representation of the second participant in the livevideo communication user interface; and changing a size and/or positionof the representation of the third participant in the live videocommunication user interface.

In some embodiments, a transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a live video communicationuser interface that includes concurrently displaying a representation ofa first participant in a live video communication session, arepresentation of a second participant in the live video communicationsession, and a representation of a third participant in the live videocommunication session; while displaying the live video communicationuser interface, receiving data indicating that an activity level of thesecond participant in the live video communication session has increasedabove the activity level of the first participant in the live videocommunication session; and in response to receiving the data indicatingthat the activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session: changing asize and/or position of the representation of the first participant inthe live video communication user interface; changing a size and/orposition of the representation of the second participant in the livevideo communication user interface; and changing a size and/or positionof the representation of the third participant in the live videocommunication user interface.

In some embodiments, an electronic device includes a display, one ormore processors, and memory. The memory stores one or more programsconfigured to be executed by the one or more processors. The one or moreprograms including instructions for: displaying a live videocommunication user interface that includes concurrently displaying arepresentation of a first participant in a live video communicationsession, a representation of a second participant in the live videocommunication session, and a representation of a third participant inthe live video communication session; while displaying the live videocommunication user interface, receiving data indicating that an activitylevel of the second participant in the live video communication sessionhas increased above the activity level of the first participant in thelive video communication session; and in response to receiving the dataindicating that the activity level of the second participant in the livevideo communication session has increased above the activity level ofthe first participant in the live video communication session: changinga size and/or position of the representation of the first participant inthe live video communication user interface; changing a size and/orposition of the representation of the second participant in the livevideo communication user interface; and changing a size and/or positionof the representation of the third participant in the live videocommunication user interface.

In some embodiments, an electronic device includes: a display; means fordisplaying a live video communication user interface that includesconcurrently displaying a representation of a first participant in alive video communication session, a representation of a secondparticipant in the live video communication session, and arepresentation of a third participant in the live video communicationsession; means for, while displaying the live video communication userinterface, receiving data indicating that an activity level of thesecond participant in the live video communication session has increasedabove the activity level of the first participant in the live videocommunication session; and means responsive to receiving the dataindicating that the activity level of the second participant in the livevideo communication session has increased above the activity level ofthe first participant in the live video communication session for:changing a size and/or position of the representation of the firstparticipant in the live video communication user interface; changing asize and/or position of the representation of the second participant inthe live video communication user interface; and changing a size and/orposition of the representation of the third participant in the livevideo communication user interface.

In some embodiments, a method is performed at an electronic device witha display. The method may include: displaying a messaging user interfaceof a messaging application, the messaging user interface including: amessage region including a plurality of messages between participants ina message conversation that includes three or more participants; and anaffordance separate from the message region for starting a live videocommunication session in a live video communication application;detecting an input corresponding to selection of the affordance; and inresponse to detecting the input corresponding to selection of theaffordance, initiating a process for starting the live videocommunication session in the live video communication application, thelive video communication session including the participants of themessage conversation.

In some embodiments, a non-transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a messaging user interfaceof a messaging application, the messaging user interface including: amessage region including a plurality of messages between participants ina message conversation that includes three or more participants; and anaffordance separate from the message region for starting a live videocommunication session in a live video communication application;detecting an input corresponding to selection of the affordance; and inresponse to detecting the input corresponding to selection of theaffordance, initiating a process for starting the live videocommunication session in the live video communication application, thelive video communication session including the participants of themessage conversation.

In some embodiments, a transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a messaging user interfaceof a messaging application, the messaging user interface including: amessage region including a plurality of messages between participants ina message conversation that includes three or more participants; and anaffordance separate from the message region for starting a live videocommunication session in a live video communication application;detecting an input corresponding to selection of the affordance; and inresponse to detecting the input corresponding to selection of theaffordance, initiating a process for starting the live videocommunication session in the live video communication application, thelive video communication session including the participants of themessage conversation.

In some embodiments, an electronic device includes a display, one ormore processors, and memory. The memory stores one or more programsconfigured to be executed by the one or more processors. The one or moreprograms including instructions for: displaying a messaging userinterface of a messaging application, the messaging user interfaceincluding: a message region including a plurality of messages betweenparticipants in a message conversation that includes three or moreparticipants; and an affordance separate from the message region forstarting a live video communication session in a live videocommunication application; detecting an input corresponding to selectionof the affordance; and in response to detecting the input correspondingto selection of the affordance, initiating a process for starting thelive video communication session in the live video communicationapplication, the live video communication session including theparticipants of the message conversation.

In some embodiments, an electronic device includes: a display; means fordisplaying a messaging user interface of a messaging application, themessaging user interface including: a message region including aplurality of messages between participants in a message conversationthat includes three or more participants; and an affordance separatefrom the message region for starting a live video communication sessionin a live video communication application; mean for detecting an inputcorresponding to selection of the affordance; and means responsive todetecting the input corresponding to selection of the affordance, forinitiating a process for starting the live video communication sessionin the live video communication application, the live videocommunication session including the participants of the messageconversation.

In some embodiments, a method is performed at an electronic device witha display. The method may include: displaying a live video communicationuser interface of a live video communication application, the live videocommunication user interface including: a plurality of participantaffordances corresponding to respective one or more participants, theplurality of participant affordances including a first participantaffordance corresponding to a group of three or more participants; and anew session affordance for starting a new live video communicationsession; detecting an input that corresponds to a request to initiate aprocess of starting a new live video communication session; and inresponse to detecting the input: in accordance with the inputcorresponding to selection of the first participant affordance,initiating a process of starting new live video communication sessionthat includes the group of three or more participants; and in accordancewith the input corresponding to selection of the new session affordance,initiating a process for selecting two or more participants and startinga new live video communication session that includes a participantassociated with the device and the two or more selected participants.

In some embodiments, a non-transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a live video communicationuser interface of a live video communication application, the live videocommunication user interface including: a plurality of participantaffordances corresponding to respective one or more participants, theplurality of participant affordances including a first participantaffordance corresponding to a group of three or more participants; and anew session affordance for starting a new live video communicationsession; detecting an input that corresponds to a request to initiate aprocess of starting a new live video communication session; and inresponse to detecting the input: in accordance with the inputcorresponding to selection of the first participant affordance,initiating a process of starting new live video communication sessionthat includes the group of three or more participants; and in accordancewith the input corresponding to selection of the new session affordance,initiating a process for selecting two or more participants and startinga new live video communication session that includes a participantassociated with the device and the two or more selected participants.

In some embodiments, a transitory computer-readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display. The one or moreprograms include instructions for: displaying a live video communicationuser interface of a live video communication application, the live videocommunication user interface including: a plurality of participantaffordances corresponding to respective one or more participants, theplurality of participant affordances including a first participantaffordance corresponding to a group of three or more participants; and anew session affordance for starting a new live video communicationsession; detecting an input that corresponds to a request to initiate aprocess of starting a new live video communication session; and inresponse to detecting the input: in accordance with the inputcorresponding to selection of the first participant affordance,initiating a process of starting new live video communication sessionthat includes the group of three or more participants; and in accordancewith the input corresponding to selection of the new session affordance,initiating a process for selecting two or more participants and startinga new live video communication session that includes a participantassociated with the device and the two or more selected participants.

In some embodiments, an electronic device includes a display, one ormore processors, and memory. The memory stores one or more programsconfigured to be executed by the one or more processors. The one or moreprograms including instructions for: displaying a live videocommunication user interface of a live video communication application,the live video communication user interface including: a plurality ofparticipant affordances corresponding to respective one or moreparticipants, the plurality of participant affordances including a firstparticipant affordance corresponding to a group of three or moreparticipants; and a new session affordance for starting a new live videocommunication session; detecting an input that corresponds to a requestto initiate a process of starting a new live video communicationsession; and in response to detecting the input: in accordance with theinput corresponding to selection of the first participant affordance,initiating a process of starting new live video communication sessionthat includes the group of three or more participants; and in accordancewith the input corresponding to selection of the new session affordance,initiating a process for selecting two or more participants and startinga new live video communication session that includes a participantassociated with the device and the two or more selected participants.

In some embodiments, an electronic device includes: a display; means fordisplaying a live video communication user interface of a live videocommunication application, the live video communication user interfaceincluding: a plurality of participant affordances corresponding torespective one or more participants, the plurality of participantaffordances including a first participant affordance corresponding to agroup of three or more participants; and a new session affordance forstarting a new live video communication session; means for detecting aninput that corresponds to a request to initiate a process of starting anew live video communication session; and means responsive to detectingthe input for: in accordance with the input corresponding to selectionof the first participant affordance, initiating a process of startingnew live video communication session that includes the group of three ormore participants; and in accordance with the input corresponding toselection of the new session affordance, initiating a process forselecting two or more participants and starting a new live videocommunication session that includes a participant associated with thedevice and the two or more selected participants.

Executable instructions for performing these functions are, optionally,included in a non-transitory computer-readable storage medium or othercomputer program product configured for execution by one or moreprocessors. Executable instructions for performing these functions are,optionally, included in a transitory computer-readable storage medium orother computer program product configured for execution by one or moreprocessors.

Thus, devices are provided with faster, more efficient methods andinterfaces for live communication involving multiple participants,thereby increasing the effectiveness, efficiency, and user satisfactionwith such devices. Such methods and interfaces may complement or replaceother methods for live communication involving multiple participants.

DESCRIPTION OF THE FIGURES

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction devicewith a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device in accordance with someembodiments.

FIG. 4B illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIG. 5A illustrates a personal electronic device in accordance with someembodiments.

FIG. 5B is a block diagram illustrating a personal electronic device inaccordance with some embodiments.

FIGS. 5C-5D illustrate exemplary components of a personal electronicdevice having a touch-sensitive display and intensity sensors inaccordance with some embodiments.

FIGS. 5E-5H illustrate exemplary components and user interfaces of apersonal electronic device in accordance with some embodiments.

FIGS. 6A-6AF illustrate exemplary user interfaces in accordance withsome embodiments.

FIGS. 7A-7F illustrate an exemplary method in accordance with someembodiments.

FIGS. 8A-8BK illustrate exemplary user interfaces in accordance withsome embodiments.

FIGS. 9A-9K illustrate an exemplary method in accordance with someembodiments.

FIGS. 10A-10S illustrate exemplary user interfaces in accordance withsome embodiments.

FIGS. 11A-11F illustrate an exemplary method in accordance with someembodiments.

FIGS. 12A-12N illustrate exemplary user interfaces in accordance withsome embodiments.

FIGS. 13A-13D illustrate an exemplary method in accordance with someembodiments.

FIGS. 14A-14I illustrate exemplary user interfaces in accordance withsome embodiments.

FIGS. 15A-15C illustrate an exemplary method in accordance with someembodiments.

DESCRIPTION OF EMBODIMENTS

The following description sets forth exemplary methods, parameters, andthe like. It should be recognized, however, that such description is notintended as a limitation on the scope of the present disclosure but isinstead provided as a description of exemplary embodiments.

There is a need for electronic devices that provide efficient methodsand interfaces for live communication involving multiple participants.Such techniques can reduce the cognitive burden on a user whoparticipates in live communication involving multiple participants,thereby enhancing productivity. Further, such techniques can reduceprocessor and battery power otherwise wasted on redundant user inputs.

Below, FIGS. 1A-1B, 2, 3, 4A-4B, and 5A-5H provide a description ofexemplary devices for performing the techniques for a live communicationsession between multiple participants. FIGS. 6A-6AF illustrate exemplaryuser interfaces for managing event notifications. FIGS. 7A-7F are a flowdiagram illustrating methods of a live communication session betweenmultiple participants in accordance with some embodiments. The userinterfaces in FIGS. 6A-6AF are used to illustrate the processesdescribed below, including the processes in FIGS. 7A-7F.

FIGS. 8A-8BK illustrate exemplary user interfaces for a livecommunication session between multiple participants. FIGS. 9A-9K are aflow diagram illustrating methods of performing a live communicationsession between multiple participants in accordance with someembodiments. The user interfaces in FIGS. 8A-8BK are used to illustratethe processes described below, including the processes in FIGS. 9A-9K.

FIGS. 10A-10S illustrate exemplary user interfaces for a livecommunication session between multiple participants. FIGS. 11A-11F are aflow diagram illustrating methods of performing a live communicationsession between multiple participants in accordance with someembodiments. The user interfaces in FIGS. 10A-10S are used to illustratethe processes described below, including the processes in FIGS. 11A-11F.

FIGS. 12A-12N illustrate exemplary user interfaces for a livecommunication session between multiple participants. FIGS. 13A-13D are aflow diagram illustrating methods of performing a live communicationsession between multiple participants in accordance with someembodiments. The user interfaces in FIGS. 12A-12N are used to illustratethe processes described below, including the processes in FIGS. 13A-13D.

FIGS. 14A-14I illustrate exemplary user interfaces for a livecommunication session between multiple participants. FIGS. 15A-15C are aflow diagram illustrating methods of performing a live communicationsession between multiple participants in accordance with someembodiments. The user interfaces in FIGS. 14A-14I are used to illustratethe processes described below, including the processes in FIGS. 15A-15C.

Although the following description uses terms “first,” “second,” etc. todescribe various elements, these elements should not be limited by theterms. These terms are only used to distinguish one element fromanother. For example, a first touch could be termed a second touch, and,similarly, a second touch could be termed a first touch, withoutdeparting from the scope of the various described embodiments. The firsttouch and the second touch are both touches, but they are not the sametouch.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

The term “if” is, optionally, construed to mean “when” or “upon” or “inresponse to determining” or “in response to detecting,” depending on thecontext. Similarly, the phrase “if it is determined” or “if [a statedcondition or event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Exemplary embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, California Other portableelectronic devices, such as laptops or tablet computers withtouch-sensitive surfaces (e.g., touch screen displays and/or touchpads),are, optionally, used. It should also be understood that, in someembodiments, the device is not a portable communications device, but isa desktop computer with a touch-sensitive surface (e.g., a touch screendisplay and/or a touchpad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse, and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, and/or a digital videoplayer application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable devices withtouch-sensitive displays. FIG. 1A is a block diagram illustratingportable multifunction device 100 with touch-sensitive display system112 in accordance with some embodiments. Touch-sensitive display 112 issometimes called a “touch screen” for convenience and is sometimes knownas or called a “touch-sensitive display system.” Device 100 includesmemory 102 (which optionally includes one or more computer-readablestorage mediums), memory controller 122, one or more processing units(CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry110, speaker 111, microphone 113, input/output (I/O) subsystem 106,other input control devices 116, and external port 124. Device 100optionally includes one or more optical sensors 164. Device 100optionally includes one or more contact intensity sensors 165 fordetecting intensity of contacts on device 100 (e.g., a touch-sensitivesurface such as touch-sensitive display system 112 of device 100).Device 100 optionally includes one or more tactile output generators 167for generating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact) on thetouch-sensitive surface, or to a substitute (proxy) for the force orpressure of a contact on the touch-sensitive surface. The intensity of acontact has a range of values that includes at least four distinctvalues and more typically includes hundreds of distinct values (e.g., atleast 256). Intensity of a contact is, optionally, determined (ormeasured) using various approaches and various sensors or combinationsof sensors. For example, one or more force sensors underneath oradjacent to the touch-sensitive surface are, optionally, used to measureforce at various points on the touch-sensitive surface. In someimplementations, force measurements from multiple force sensors arecombined (e.g., a weighted average) to determine an estimated force of acontact. Similarly, a pressure-sensitive tip of a stylus is, optionally,used to determine a pressure of the stylus on the touch-sensitivesurface. Alternatively, the size of the contact area detected on thetouch-sensitive surface and/or changes thereto, the capacitance of thetouch-sensitive surface proximate to the contact and/or changes thereto,and/or the resistance of the touch-sensitive surface proximate to thecontact and/or changes thereto are, optionally, used as a substitute forthe force or pressure of the contact on the touch-sensitive surface. Insome implementations, the substitute measurements for contact force orpressure are used directly to determine whether an intensity thresholdhas been exceeded (e.g., the intensity threshold is described in unitscorresponding to the substitute measurements). In some implementations,the substitute measurements for contact force or pressure are convertedto an estimated force or pressure, and the estimated force or pressureis used to determine whether an intensity threshold has been exceeded(e.g., the intensity threshold is a pressure threshold measured in unitsof pressure). Using the intensity of a contact as an attribute of a userinput allows for user access to additional device functionality that mayotherwise not be accessible by the user on a reduced-size device withlimited real estate for displaying affordances (e.g., on atouch-sensitive display) and/or receiving user input (e.g., via atouch-sensitive display, a touch-sensitive surface, or aphysical/mechanical control such as a knob or a button).

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as a “down click” or “up click” even when there is no movement of aphysical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user.

It should be appreciated that device 100 is only one example of aportable multifunction device, and that device 100 optionally has moreor fewer components than shown, optionally combines two or morecomponents, or optionally has a different configuration or arrangementof the components. The various components shown in FIG. 1A areimplemented in hardware, software, or a combination of both hardware andsoftware, including one or more signal processing and/orapplication-specific integrated circuits.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Memory controller 122optionally controls access to memory 102 by other components of device100.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data. In some embodiments, peripheralsinterface 118, CPU 120, and memory controller 122 are, optionally,implemented on a single chip, such as chip 104. In some otherembodiments, they are, optionally, implemented on separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The RF circuitry 108optionally includes well-known circuitry for detecting near fieldcommunication (NFC) fields, such as by a short-range communicationradio. The wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity(Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n,and/or IEEE 802.11ac), voice over Internet Protocol (VoIP), Wi-MAX, aprotocol for e-mail (e.g., Internet message access protocol (IMAP)and/or post office protocol (POP)), instant messaging (e.g., extensiblemessaging and presence protocol (XMPP), Session Initiation Protocol forInstant Messaging and Presence Leveraging Extensions (SIMPLE), InstantMessaging and Presence Service (IMPS)), and/or Short Message Service(SMS), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 111 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2 ). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch screen 112 and other input control devices 116, to peripheralsinterface 118. I/O subsystem 106 optionally includes display controller156, optical sensor controller 158, depth camera controller 169,intensity sensor controller 159, haptic feedback controller 161, and oneor more input controllers 160 for other input or control devices. Theone or more input controllers 160 receive/send electrical signalsfrom/to other input control devices 116. The other input control devices116 optionally include physical buttons (e.g., push buttons, rockerbuttons, etc.), dials, slider switches, joysticks, click wheels, and soforth. In some alternate embodiments, input controller(s) 160 are,optionally, coupled to any (or none) of the following: a keyboard, aninfrared port, a USB port, and a pointer device such as a mouse. The oneor more buttons (e.g., 208, FIG. 2 ) optionally include an up/downbutton for volume control of speaker 111 and/or microphone 113. The oneor more buttons optionally include a push button (e.g., 206, FIG. 2 ).

A quick press of the push button optionally disengages a lock of touchscreen 112 or optionally begins a process that uses gestures on thetouch screen to unlock the device, as described in U.S. patentapplication Ser. No. 11/322,549, “Unlocking a Device by PerformingGestures on an Unlock Image,” filed Dec. 23, 2005, U.S. Pat. No.7,657,849, which is hereby incorporated by reference in its entirety. Alonger press of the push button (e.g., 206) optionally turns power todevice 100 on or off. The functionality of one or more of the buttonsare, optionally, user-customizable. Touch screen 112 is used toimplement virtual or soft buttons and one or more soft keyboards.

Touch-sensitive display 112 provides an input interface and an outputinterface between the device and a user. Display controller 156 receivesand/or sends electrical signals from/to touch screen 112. Touch screen112 displays visual output to the user. The visual output optionallyincludes graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output optionally corresponds to user-interface objects.

Touch screen 112 has a touch-sensitive surface, sensor, or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. Touch screen 112 and display controller 156 (along with anyassociated modules and/or sets of instructions in memory 102) detectcontact (and any movement or breaking of the contact) on touch screen112 and convert the detected contact into interaction withuser-interface objects (e.g., one or more soft keys, icons, web pages,or images) that are displayed on touch screen 112. In an exemplaryembodiment, a point of contact between touch screen 112 and the usercorresponds to a finger of the user.

Touch screen 112 optionally uses LCD (liquid crystal display)technology, LPD (light emitting polymer display) technology, or LED(light emitting diode) technology, although other display technologiesare used in other embodiments. Touch screen 112 and display controller156 optionally detect contact and any movement or breaking thereof usingany of a plurality of touch sensing technologies now known or laterdeveloped, including but not limited to capacitive, resistive, infrared,and surface acoustic wave technologies, as well as other proximitysensor arrays or other elements for determining one or more points ofcontact with touch screen 112. In an exemplary embodiment, projectedmutual capacitance sensing technology is used, such as that found in theiPhone® and iPod Touch® from Apple Inc. of Cupertino, California.

A touch-sensitive display in some embodiments of touch screen 112 is,optionally, analogous to the multi-touch sensitive touchpads describedin the following U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat.No. 6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932(Westerman), and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference in its entirety. However,touch screen 112 displays visual output from device 100, whereastouch-sensitive touchpads do not provide visual output.

A touch-sensitive display in some embodiments of touch screen 112 isdescribed in the following applications: (1) U.S. patent applicationSer. No. 11/381,313, “Multipoint Touch Surface Controller,” filed May 2,2006; (2) U.S. patent application Ser. No. 10/840,862, “MultipointTouchscreen,” filed May 6, 2004; (3) U.S. patent application Ser. No.10/903,964, “Gestures For Touch Sensitive Input Devices,” filed Jul. 30,2004; (4) U.S. patent application Ser. No. 11/048,264, “Gestures ForTouch Sensitive Input Devices,” filed Jan. 31, 2005; (5) U.S. patentapplication Ser. No. 11/038,590, “Mode-Based Graphical User InterfacesFor Touch Sensitive Input Devices,” filed Jan. 18, 2005; (6) U.S. patentapplication Ser. No. 11/228,758, “Virtual Input Device Placement On ATouch Screen User Interface,” filed Sep. 16, 2005; (7) U.S. patentapplication Ser. No. 11/228,700, “Operation Of A Computer With A TouchScreen Interface,” filed Sep. 16, 2005; (8) U.S. patent application Ser.No. 11/228,737, “Activating Virtual Keys Of A Touch-Screen VirtualKeyboard,” filed Sep. 16, 2005; and (9) U.S. patent application Ser. No.11/367,749, “Multi-Functional Hand-Held Device,” filed Mar. 3, 2006. Allof these applications are incorporated by reference herein in theirentirety.

Touch screen 112 optionally has a video resolution in excess of 100 dpi.In some embodiments, the touch screen has a video resolution ofapproximately 160 dpi. The user optionally makes contact with touchscreen 112 using any suitable object or appendage, such as a stylus, afinger, and so forth. In some embodiments, the user interface isdesigned to work primarily with finger-based contacts and gestures,which can be less precise than stylus-based input due to the larger areaof contact of a finger on the touch screen. In some embodiments, thedevice translates the rough finger-based input into a precisepointer/cursor position or command for performing the actions desired bythe user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad is, optionally, a touch-sensitive surface that isseparate from touch screen 112 or an extension of the touch-sensitivesurface formed by the touch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled to optical sensor controller 158in I/O subsystem 106. Optical sensor 164 optionally includescharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 164 receives light from theenvironment, projected through one or more lenses, and converts thelight to data representing an image. In conjunction with imaging module143 (also called a camera module), optical sensor 164 optionallycaptures still images or video. In some embodiments, an optical sensoris located on the back of device 100, opposite touch screen display 112on the front of the device so that the touch screen display is enabledfor use as a viewfinder for still and/or video image acquisition. Insome embodiments, an optical sensor is located on the front of thedevice so that the user's image is, optionally, obtained for videoconferencing while the user views the other video conferenceparticipants on the touch screen display. In some embodiments, theposition of optical sensor 164 can be changed by the user (e.g., byrotating the lens and the sensor in the device housing) so that a singleoptical sensor 164 is used along with the touch screen display for bothvideo conferencing and still and/or video image acquisition.

Device 100 optionally also includes one or more depth camera sensors175. FIG. 1A shows a depth camera sensor coupled to depth cameracontroller 169 in I/O subsystem 106. Depth camera sensor 175 receivesdata from the environment to create a three dimensional model of anobject (e.g., a face) within a scene from a viewpoint (e.g., a depthcamera sensor). In some embodiments, in conjunction with imaging module143 (also called a camera module), depth camera sensor 175 is optionallyused to determine a depth map of different portions of an image capturedby the imaging module 143. In some embodiments, a depth camera sensor islocated on the front of device 100 so that the user's image with depthinformation is, optionally, obtained for video conferencing while theuser views the other video conference participants on the touch screendisplay and to capture selfies with depth map data. In some embodiments,the depth camera sensor 175 is located on the back of device, or on theback and the front of the device 100. In some embodiments, the positionof depth camera sensor 175 can be changed by the user (e.g., by rotatingthe lens and the sensor in the device housing) so that a depth camerasensor 175 is used along with the touch screen display for both videoconferencing and still and/or video image acquisition.

In some embodiments, a depth map (e.g., depth map image) containsinformation (e.g., values) that relates to the distance of objects in ascene from a viewpoint (e.g., a camera, an optical sensor, a depthcamera sensor). In one embodiment of a depth map, each depth pixeldefines the position in the viewpoint's Z-axis where its correspondingtwo-dimensional pixel is located. In some embodiments, a depth map iscomposed of pixels wherein each pixel is defined by a value (e.g.,0-255). For example, the “0” value represents pixels that are located atthe most distant place in a “three dimensional” scene and the “255”value represents pixels that are located closest to a viewpoint (e.g., acamera, an optical sensor, a depth camera sensor) in the “threedimensional” scene. In other embodiments, a depth map represents thedistance between an object in a scene and the plane of the viewpoint. Insome embodiments, the depth map includes information about the relativedepth of various features of an object of interest in view of the depthcamera (e.g., the relative depth of eyes, nose, mouth, ears of a user'sface). In some embodiments, the depth map includes information thatenables the device to determine contours of the object of interest in az direction.

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled tointensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor 165 optionally includes one or more piezoresistive strain gauges,capacitive force sensors, electric force sensors, piezoelectric forcesensors, optical force sensors, capacitive touch-sensitive surfaces, orother intensity sensors (e.g., sensors used to measure the force (orpressure) of a contact on a touch-sensitive surface). Contact intensitysensor 165 receives contact intensity information (e.g., pressureinformation or a proxy for pressure information) from the environment.In some embodiments, at least one contact intensity sensor is collocatedwith, or proximate to, a touch-sensitive surface (e.g., touch-sensitivedisplay system 112). In some embodiments, at least one contact intensitysensor is located on the back of device 100, opposite touch screendisplay 112, which is located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118.Alternately, proximity sensor 166 is, optionally, coupled to inputcontroller 160 in I/O subsystem 106. Proximity sensor 166 optionallyperforms as described in U.S. patent application Ser. No. 11/241,839,“Proximity Detector In Handheld Device”; Ser. No. 11/240,788, “ProximityDetector In Handheld Device”; Ser. No. 11/620,702, “Using Ambient LightSensor To Augment Proximity Sensor Output”; Ser. No. 11/586,862,“Automated Response To And Sensing Of User Activity In PortableDevices”; and Ser. No. 11/638,251, “Methods And Systems For AutomaticConfiguration Of Peripherals,” which are hereby incorporated byreference in their entirety. In some embodiments, the proximity sensorturns off and disables touch screen 112 when the multifunction device isplaced near the user's ear (e.g., when the user is making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled tohaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator 167 optionally includes one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). Contact intensity sensor 165 receives tactile feedbackgeneration instructions from haptic feedback module 133 and generatestactile outputs on device 100 that are capable of being sensed by a userof device 100. In some embodiments, at least one tactile outputgenerator is collocated with, or proximate to, a touch-sensitive surface(e.g., touch-sensitive display system 112) and, optionally, generates atactile output by moving the touch-sensitive surface vertically (e.g.,in/out of a surface of device 100) or laterally (e.g., back and forth inthe same plane as a surface of device 100). In some embodiments, atleast one tactile output generator sensor is located on the back ofdevice 100, opposite touch screen display 112, which is located on thefront of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled to peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. Accelerometer 168 optionallyperforms as described in U.S. Patent Publication No. 20050190059,“Acceleration-based Theft Detection System for Portable ElectronicDevices,” and U.S. Patent Publication No. 20060017692, “Methods AndApparatuses For Operating A Portable Device Based On An Accelerometer,”both of which are incorporated by reference herein in their entirety. Insome embodiments, information is displayed on the touch screen displayin a portrait view or a landscape view based on an analysis of datareceived from the one or more accelerometers. Device 100 optionallyincludes, in addition to accelerometer(s) 168, a magnetometer and a GPS(or GLONASS or other global navigation system) receiver for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, text input module (or setof instructions) 134, Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or sets of instructions) 136.Furthermore, in some embodiments, memory 102 (FIG. 1A) or 370 (FIG. 3 )stores device/global internal state 157, as shown in FIGS. 1A and 3 .Device/global internal state 157 includes one or more of: activeapplication state, indicating which applications, if any, are currentlyactive; display state, indicating what applications, views or otherinformation occupy various regions of touch screen display 112; sensorstate, including information obtained from the device's various sensorsand input control devices 116; and location information concerning thedevice's location and/or attitude.

Operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with, the30-pin connector used on iPod® (trademark of Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen112 (in conjunction with display controller 156) and othertouch-sensitive devices (e.g., a touchpad or physical click wheel).Contact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact), determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). In some embodiments,contact/motion module 130 and display controller 156 detect contact on atouchpad.

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments, at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch screen display can be set to any of alarge range of predefined threshold values without changing the trackpador touch screen display hardware. Additionally, in some implementations,a user of the device is provided with software settings for adjustingone or more of the set of intensity thresholds (e.g., by adjustingindividual intensity thresholds and/or by adjusting a plurality ofintensity thresholds at once with a system-level click “intensity”parameter).

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns (e.g., different motions, timings, and/or intensities ofdetected contacts). Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (liftoff) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (liftoff) event.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch screen 112 or other display,including components for changing the visual impact (e.g., brightness,transparency, saturation, contrast, or other visual property) ofgraphics that are displayed. As used herein, the term “graphics”includes any object that can be displayed to a user, including, withoutlimitation, text, web pages, icons (such as user-interface objectsincluding soft keys), digital images, videos, animations, and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions used by tactile output generator(s) 167 toproduce tactile outputs at one or more locations on device 100 inresponse to user interactions with device 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone module138 for use in location-based dialing; to camera module 143 aspicture/video metadata; and to applications that provide location-basedservices such as weather widgets, local yellow page widgets, andmap/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   Contacts module 137 (sometimes called an address book or contact        list);    -   Telephone module 138;    -   Video conference module 139;    -   E-mail client module 140;    -   Instant messaging (IM) module 141;    -   Workout support module 142;    -   Camera module 143 for still and/or video images;    -   Image management module 144;    -   Video player module;    -   Music player module;    -   Browser module 147;    -   Calendar module 148;    -   Widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   Widget creator module 150 for making user-created widgets 149-6;    -   Search module 151;    -   Video and music player module 152, which merges video player        module and music player module;    -   Notes module 153;    -   Map module 154; and/or    -   Online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, contacts module 137 are, optionally, used to manage an address bookor contact list (e.g., stored in application internal state 192 ofcontacts module 137 in memory 102 or memory 370), including: addingname(s) to the address book; deleting name(s) from the address book;associating telephone number(s), e-mail address(es), physicaladdress(es) or other information with a name; associating an image witha name; categorizing and sorting names; providing telephone numbers ore-mail addresses to initiate and/or facilitate communications bytelephone 138, video conference module 139, e-mail 140, or IM 141; andso forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact/motionmodule 130, graphics module 132, and text input module 134, telephonemodule 138 are optionally, used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in contacts module 137, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation, anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact/motion module 130, graphicsmodule 132, text input module 134, contacts module 137, and telephonemodule 138, video conference module 139 includes executable instructionsto initiate, conduct, and terminate a video conference between a userand one or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, e-mail client module 140 includes executableinstructions to create, send, receive, and manage e-mail in response touser instructions. In conjunction with image management module 144,e-mail client module 140 makes it very easy to create and send e-mailswith still or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, the instant messaging module 141 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (for example, using a Short Message Service(SMS) or Multimedia Message Service (MMS) protocol for telephony-basedinstant messages or using XMPP, SIMPLE, or IMPS for Internet-basedinstant messages), to receive instant messages, and to view receivedinstant messages. In some embodiments, transmitted and/or receivedinstant messages optionally include graphics, photos, audio files, videofiles and/or other attachments as are supported in an MMS and/or anEnhanced Messaging Service (EMS). As used herein, “instant messaging”refers to both telephony-based messages (e.g., messages sent using SMSor MMS) and Internet-based messages (e.g., messages sent using XMPP,SIMPLE, or IMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, map module 154, and music playermodule, workout support module 142 includes executable instructions tocreate workouts (e.g., with time, distance, and/or calorie burninggoals); communicate with workout sensors (sports devices); receiveworkout sensor data; calibrate sensors used to monitor a workout; selectand play music for a workout; and display, store, and transmit workoutdata.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact/motion module 130,graphics module 132, and image management module 144, camera module 143includes executable instructions to capture still images or video(including a video stream) and store them into memory 102, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 102.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, text input module 134,and camera module 143, image management module 144 includes executableinstructions to arrange, modify (e.g., edit), or otherwise manipulate,label, delete, present (e.g., in a digital slide show or album), andstore still and/or video images.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, browser module 147 includes executable instructions tobrowse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, e-mail client module 140, and browser module 147,calendar module 148 includes executable instructions to create, display,modify, and store calendars and data associated with calendars (e.g.,calendar entries, to-do lists, etc.) in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, widget modules 149 aremini-applications that are, optionally, downloaded and used by a user(e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, the widget creator module 150are, optionally, used by a user to create widgets (e.g., turning auser-specified portion of a web page into a widget).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, search module 151 includes executable instructions to search fortext, music, sound, image, video, and/or other files in memory 102 thatmatch one or more search criteria (e.g., one or more user-specifiedsearch terms) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, and browser module 147, video and musicplayer module 152 includes executable instructions that allow the userto download and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present, or otherwise play back videos (e.g.,on touch screen 112 or on an external, connected display via externalport 124). In some embodiments, device 100 optionally includes thefunctionality of an MP3 player, such as an iPod (trademark of AppleInc.).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, notes module 153 includes executable instructions to create andmanage notes, to-do lists, and the like in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, and browser module 147, map module 154are, optionally, used to receive, display, modify, and store maps anddata associated with maps (e.g., driving directions, data on stores andother points of interest at or near a particular location, and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, text input module 134, e-mail clientmodule 140, and browser module 147, online video module 155 includesinstructions that allow the user to access, browse, receive (e.g., bystreaming and/or download), play back (e.g., on the touch screen or onan external, connected display via external port 124), send an e-mailwith a link to a particular online video, and otherwise manage onlinevideos in one or more file formats, such as H.264. In some embodiments,instant messaging module 141, rather than e-mail client module 140, isused to send a link to a particular online video. Additional descriptionof the online video application can be found in U.S. Provisional PatentApplication No. 60/936,562, “Portable Multifunction Device, Method, andGraphical User Interface for Playing Online Videos,” filed Jun. 20,2007, and U.S. patent application Ser. No. 11/968,067, “PortableMultifunction Device, Method, and Graphical User Interface for PlayingOnline Videos,” filed Dec. 31, 2007, the contents of which are herebyincorporated by reference in their entirety.

Each of the above-identified modules and applications corresponds to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (e.g., sets of instructions) need notbe implemented as separate software programs, procedures, or modules,and thus various subsets of these modules are, optionally, combined orotherwise rearranged in various embodiments. For example, video playermodule is, optionally, combined with music player module into a singlemodule (e.g., video and music player module 152, FIG. 1A). In someembodiments, memory 102 optionally stores a subset of the modules anddata structures identified above. Furthermore, memory 102 optionallystores additional modules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device100, the number of physical input control devices (such as push buttons,dials, and the like) on device 100 is, optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (FIG. 1A) or 370 (FIG. 3 ) includes event sorter 170 (e.g.,in operating system 126) and a respective application 136-1 (e.g., anyof the aforementioned applications 137-151, 155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch-sensitivedisplay 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display 112, as part of a multi-touchgesture). Peripherals interface 118 transmits information it receivesfrom I/O subsystem 106 or a sensor, such as proximity sensor 166,accelerometer(s) 168, and/or microphone 113 (through audio circuitry110). Information that peripherals interface 118 receives from I/Osubsystem 106 includes information from touch-sensitive display 112 or atouch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripherals interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more viewswhen touch-sensitive display 112 displays more than one view. Views aremade up of controls and other elements that a user can see on thedisplay.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (e.g., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule 172, the hit view typically receives all sub-events related tothe same touch or input source for which it was identified as the hitview.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysical location of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver 182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit or a higher level object from which application 136-1 inheritsmethods and other properties. In some embodiments, a respective eventhandler 190 includes one or more of: data updater 176, object updater177, GUI updater 178, and/or event data 179 received from event sorter170. Event handler 190 optionally utilizes or calls data updater 176,object updater 177, or GUI updater 178 to update the applicationinternal state 192. Alternatively, one or more of the application views191 include one or more respective event handlers 190. Also, in someembodiments, one or more of data updater 176, object updater 177, andGUI updater 178 are included in a respective application view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170 and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event (187) include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first liftoff (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second liftoff (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay 112, and liftoff of the touch (touch end). In some embodiments,the event also includes information for one or more associated eventhandlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display 112, when a touch is detected on touch-sensitivedisplay 112, event comparator 184 performs a hit test to determine whichof the three user-interface objects is associated with the touch(sub-event). If each displayed object is associated with a respectiveevent handler 190, the event comparator uses the result of the hit testto determine which event handler 190 should be activated. For example,event comparator 184 selects an event handler associated with thesub-event and the object triggering the hit test.

In some embodiments, the definition for a respective event (187) alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater177 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays also applies toother forms of user inputs to operate multifunction devices 100 withinput devices, not all of which are initiated on touch screens. Forexample, mouse movement and mouse button presses, optionally coordinatedwith single or multiple keyboard presses or holds; contact movementssuch as taps, drags, scrolls, etc. on touchpads; pen stylus inputs;movement of the device; oral instructions; detected eye movements;biometric inputs; and/or any combination thereof are optionally utilizedas inputs corresponding to sub-events which define an event to berecognized.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen 112 in accordance with some embodiments. The touch screenoptionally displays one or more graphics within user interface (UI) 200.In this embodiment, as well as others described below, a user is enabledto select one or more of the graphics by making a gesture on thegraphics, for example, with one or more fingers 202 (not drawn to scalein the figure) or one or more styluses 203 (not drawn to scale in thefigure). In some embodiments, selection of one or more graphics occurswhen the user breaks contact with the one or more graphics. In someembodiments, the gesture optionally includes one or more taps, one ormore swipes (from left to right, right to left, upward and/or downward),and/or a rolling of a finger (from right to left, left to right, upwardand/or downward) that has made contact with device 100. In someimplementations or circumstances, inadvertent contact with a graphicdoes not select the graphic. For example, a swipe gesture that sweepsover an application icon optionally does not select the correspondingapplication when the gesture corresponding to selection is a tap.

Device 100 optionally also include one or more physical buttons, such as“home” or menu button 204. As described previously, menu button 204 is,optionally, used to navigate to any application 136 in a set ofapplications that are, optionally, executed on device 100.Alternatively, in some embodiments, the menu button is implemented as asoft key in a GUI displayed on touch screen 112.

In some embodiments, device 100 includes touch screen 112, menu button204, push button 206 for powering the device on/off and locking thedevice, volume adjustment button(s) 208, subscriber identity module(SIM) card slot 210, headset jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, device 100 also acceptsverbal input for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch screen 112 and/or one or more tactile output generators 167 forgenerating tactile outputs for a user of device 100.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not be portable. In some embodiments,device 300 is a laptop computer, a desktop computer, a tablet computer,a multimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller). Device 300 typically includesone or more processing units (CPUs) 310, one or more network or othercommunications interfaces 360, memory 370, and one or more communicationbuses 320 for interconnecting these components. Communication buses 320optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.Device 300 includes input/output (I/O) interface 330 comprising display340, which is typically a touch screen display. I/O interface 330 alsooptionally includes a keyboard and/or mouse (or other pointing device)350 and touchpad 355, tactile output generator 357 for generatingtactile outputs on device 300 (e.g., similar to tactile outputgenerator(s) 167 described above with reference to FIG. 1A), sensors 359(e.g., optical, acceleration, proximity, touch-sensitive, and/or contactintensity sensors similar to contact intensity sensor(s) 165 describedabove with reference to FIG. 1A). Memory 370 includes high-speed randomaccess memory, such as DRAM, SRAM, DDR RAM, or other random access solidstate memory devices; and optionally includes non-volatile memory, suchas one or more magnetic disk storage devices, optical disk storagedevices, flash memory devices, or other non-volatile solid state storagedevices. Memory 370 optionally includes one or more storage devicesremotely located from CPU(s) 310. In some embodiments, memory 370 storesprograms, modules, and data structures analogous to the programs,modules, and data structures stored in memory 102 of portablemultifunction device 100 (FIG. 1A), or a subset thereof. Furthermore,memory 370 optionally stores additional programs, modules, and datastructures not present in memory 102 of portable multifunction device100. For example, memory 370 of device 300 optionally stores drawingmodule 380, presentation module 382, word processing module 384, websitecreation module 386, disk authoring module 388, and/or spreadsheetmodule 390, while memory 102 of portable multifunction device 100 (FIG.1A) optionally does not store these modules.

Each of the above-identified elements in FIG. 3 is, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove-identified modules corresponds to a set of instructions forperforming a function described above. The above-identified modules orprograms (e.g., sets of instructions) need not be implemented asseparate software programs, procedures, or modules, and thus varioussubsets of these modules are, optionally, combined or otherwiserearranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

Attention is now directed towards embodiments of user interfaces thatare, optionally, implemented on, for example, portable multifunctiondevice 100.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces are, optionally, implementedon device 300. In some embodiments, user interface 400 includes thefollowing elements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Icon 416 for telephone module 138, labeled “Phone,” which            optionally includes an indicator 414 of the number of missed            calls or voicemail messages;        -   Icon 418 for e-mail client module 140, labeled “Mail,” which            optionally includes an indicator 410 of the number of unread            e-mails;        -   Icon 420 for browser module 147, labeled “Browser;” and        -   Icon 422 for video and music player module 152, also            referred to as iPod (trademark of Apple Inc.) module 152,            labeled “iPod;” and    -   Icons for other applications, such as:        -   Icon 424 for IM module 141, labeled “Messages;”        -   Icon 426 for calendar module 148, labeled “Calendar;”        -   Icon 428 for image management module 144, labeled “Photos;”        -   Icon 430 for camera module 143, labeled “Camera;”        -   Icon 432 for online video module 155, labeled “Online            Video;”        -   Icon 434 for stocks widget 149-2, labeled “Stocks;”        -   Icon 436 for map module 154, labeled “Maps;”        -   Icon 438 for weather widget 149-1, labeled “Weather;”        -   Icon 440 for alarm clock widget 149-4, labeled “Clock;”        -   Icon 442 for workout support module 142, labeled “Workout            Support;”        -   Icon 444 for notes module 153, labeled “Notes;” and        -   Icon 446 for a settings application or module, labeled            “Settings,” which provides access to settings for device 100            and its various applications 136.

It should be noted that the icon labels illustrated in FIG. 4A aremerely exemplary. For example, icon 422 for video and music playermodule 152 is labeled “Music” or “Music Player.” Other labels are,optionally, used for various application icons. In some embodiments, alabel for a respective application icon includes a name of anapplication corresponding to the respective application icon. In someembodiments, a label for a particular application icon is distinct froma name of an application corresponding to the particular applicationicon.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3 ) with a touch-sensitive surface 451 (e.g., a tabletor touchpad 355, FIG. 3 ) that is separate from the display 450 (e.g.,touch screen display 112). Device 300 also, optionally, includes one ormore contact intensity sensors (e.g., one or more of sensors 359) fordetecting intensity of contacts on touch-sensitive surface 451 and/orone or more tactile output generators 357 for generating tactile outputsfor a user of device 300.

Although some of the examples that follow will be given with referenceto inputs on touch screen display 112 (where the touch-sensitive surfaceand the display are combined), in some embodiments, the device detectsinputs on a touch-sensitive surface that is separate from the display,as shown in FIG. 4B. In some embodiments, the touch-sensitive surface(e.g., 451 in FIG. 4B) has a primary axis (e.g., 452 in FIG. 4B) thatcorresponds to a primary axis (e.g., 453 in FIG. 4B) on the display(e.g., 450). In accordance with these embodiments, the device detectscontacts (e.g., 460 and 462 in FIG. 4B) with the touch-sensitive surface451 at locations that correspond to respective locations on the display(e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470).In this way, user inputs (e.g., contacts 460 and 462, and movementsthereof) detected by the device on the touch-sensitive surface (e.g.,451 in FIG. 4B) are used by the device to manipulate the user interfaceon the display (e.g., 450 in FIG. 4B) of the multifunction device whenthe touch-sensitive surface is separate from the display. It should beunderstood that similar methods are, optionally, used for other userinterfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse-based input or stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

FIG. 5A illustrates exemplary personal electronic device 500. Device 500includes body 502. In some embodiments, device 500 can include some orall of the features described with respect to devices 100 and 300 (e.g.,FIGS. 1A-4B). In some embodiments, device 500 has touch-sensitivedisplay screen 504, hereafter touch screen 504. Alternatively, or inaddition to touch screen 504, device 500 has a display and atouch-sensitive surface. As with devices 100 and 300, in someembodiments, touch screen 504 (or the touch-sensitive surface)optionally includes one or more intensity sensors for detectingintensity of contacts (e.g., touches) being applied. The one or moreintensity sensors of touch screen 504 (or the touch-sensitive surface)can provide output data that represents the intensity of touches. Theuser interface of device 500 can respond to touches based on theirintensity, meaning that touches of different intensities can invokedifferent user interface operations on device 500.

Exemplary techniques for detecting and processing touch intensity arefound, for example, in related applications: International PatentApplication Serial No. PCT/US2013/040061, titled “Device, Method, andGraphical User Interface for Displaying User Interface ObjectsCorresponding to an Application,” filed May 8, 2013, published as WIPOPublication No. WO/2013/169849, and International Patent ApplicationSerial No. PCT/US2013/069483, titled “Device, Method, and Graphical UserInterface for Transitioning Between Touch Input to Display OutputRelationships,” filed Nov. 11, 2013, published as WIPO Publication No.WO/2014/105276, each of which is hereby incorporated by reference intheir entirety.

In some embodiments, device 500 has one or more input mechanisms 506 and508. Input mechanisms 506 and 508, if included, can be physical.Examples of physical input mechanisms include push buttons and rotatablemechanisms. In some embodiments, device 500 has one or more attachmentmechanisms. Such attachment mechanisms, if included, can permitattachment of device 500 with, for example, hats, eyewear, earrings,necklaces, shirts, jackets, bracelets, watch straps, chains, trousers,belts, shoes, purses, backpacks, and so forth. These attachmentmechanisms permit device 500 to be worn by a user.

FIG. 5B depicts exemplary personal electronic device 500. In someembodiments, device 500 can include some or all of the componentsdescribed with respect to FIGS. 1A, 1B, and 3. Device 500 has bus 512that operatively couples I/O section 514 with one or more computerprocessors 516 and memory 518. I/O section 514 can be connected todisplay 504, which can have touch-sensitive component 522 and,optionally, intensity sensor 524 (e.g., contact intensity sensor). Inaddition, I/O section 514 can be connected with communication unit 530for receiving application and operating system data, using Wi-Fi,Bluetooth, near field communication (NFC), cellular, and/or otherwireless communication techniques. Device 500 can include inputmechanisms 506 and/or 508. Input mechanism 506 is, optionally, arotatable input device or a depressible and rotatable input device, forexample. Input mechanism 508 is, optionally, a button, in some examples.

Input mechanism 508 is, optionally, a microphone, in some examples.Personal electronic device 500 optionally includes various sensors, suchas GPS sensor 532, accelerometer 534, directional sensor 540 (e.g.,compass), gyroscope 536, motion sensor 538, and/or a combinationthereof, all of which can be operatively connected to I/O section 514.

Memory 518 of personal electronic device 500 can include one or morenon-transitory computer-readable storage mediums, for storingcomputer-executable instructions, which, when executed by one or morecomputer processors 516, for example, can cause the computer processorsto perform the techniques described below, including processes 700, 900,1100, 1300, and 1500 (FIGS. 7A-7F, 9A-9K, 11A-11F, 13A-13D, and15A-15C). A computer-readable storage medium can be any medium that cantangibly contain or store computer-executable instructions for use by orin connection with the instruction execution system, apparatus, ordevice. In some examples, the storage medium is a transitorycomputer-readable storage medium. In some examples, the storage mediumis a non-transitory computer-readable storage medium. The non-transitorycomputer-readable storage medium can include, but is not limited to,magnetic, optical, and/or semiconductor storages. Examples of suchstorage include magnetic disks, optical discs based on CD, DVD, orBlu-ray technologies, as well as persistent solid-state memory such asflash, solid-state drives, and the like. Personal electronic device 500is not limited to the components and configuration of FIG. 5B, but caninclude other or additional components in multiple configurations.

As used here, the term “affordance” refers to a user-interactivegraphical user interface object that is, optionally, displayed on thedisplay screen of devices 100, 300, and/or 500 (FIGS. 1A, 3, and 5A-5B).For example, an image (e.g., icon), a button, and text (e.g., hyperlink)each optionally constitute an affordance.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B)while the cursor is over a particular user interface element (e.g., abutton, window, slider, or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations that include a touch screen display(e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112in FIG. 4A) that enables direct interaction with user interface elementson the touch screen display, a detected contact on the touch screen actsas a “focus selector” so that when an input (e.g., a press input by thecontact) is detected on the touch screen display at a location of aparticular user interface element (e.g., a button, window, slider, orother user interface element), the particular user interface element isadjusted in accordance with the detected input. In some implementations,focus is moved from one region of a user interface to another region ofthe user interface without corresponding movement of a cursor ormovement of a contact on a touch screen display (e.g., by using a tabkey or arrow keys to move focus from one button to another button); inthese implementations, the focus selector moves in accordance withmovement of focus between different regions of the user interface.Without regard to the specific form taken by the focus selector, thefocus selector is generally the user interface element (or contact on atouch screen display) that is controlled by the user so as tocommunicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact, or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

As used in the specification and claims, the term “characteristicintensity” of a contact refers to a characteristic of the contact basedon one or more intensities of the contact. In some embodiments, thecharacteristic intensity is based on multiple intensity samples. Thecharacteristic intensity is, optionally, based on a predefined number ofintensity samples, or a set of intensity samples collected during apredetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10seconds) relative to a predefined event (e.g., after detecting thecontact, prior to detecting liftoff of the contact, before or afterdetecting a start of movement of the contact, prior to detecting an endof the contact, before or after detecting an increase in intensity ofthe contact, and/or before or after detecting a decrease in intensity ofthe contact). A characteristic intensity of a contact is, optionally,based on one or more of: a maximum value of the intensities of thecontact, a mean value of the intensities of the contact, an averagevalue of the intensities of the contact, a top 10 percentile value ofthe intensities of the contact, a value at the half maximum of theintensities of the contact, a value at the 90 percent maximum of theintensities of the contact, or the like. In some embodiments, theduration of the contact is used in determining the characteristicintensity (e.g., when the characteristic intensity is an average of theintensity of the contact over time). In some embodiments, thecharacteristic intensity is compared to a set of one or more intensitythresholds to determine whether an operation has been performed by auser. For example, the set of one or more intensity thresholdsoptionally includes a first intensity threshold and a second intensitythreshold. In this example, a contact with a characteristic intensitythat does not exceed the first threshold results in a first operation, acontact with a characteristic intensity that exceeds the first intensitythreshold and does not exceed the second intensity threshold results ina second operation, and a contact with a characteristic intensity thatexceeds the second threshold results in a third operation. In someembodiments, a comparison between the characteristic intensity and oneor more thresholds is used to determine whether or not to perform one ormore operations (e.g., whether to perform a respective operation orforgo performing the respective operation), rather than being used todetermine whether to perform a first operation or a second operation.

FIG. 5C illustrates detecting a plurality of contacts 552A-552E ontouch-sensitive display screen 504 with a plurality of intensity sensors524A-524D. FIG. 5C additionally includes intensity diagrams that showthe current intensity measurements of the intensity sensors 524A-524Drelative to units of intensity. In this example, the intensitymeasurements of intensity sensors 524A and 524D are each 9 units ofintensity, and the intensity measurements of intensity sensors 524B and524C are each 7 units of intensity. In some implementations, anaggregate intensity is the sum of the intensity measurements of theplurality of intensity sensors 524A-524D, which in this example is 32intensity units. In some embodiments, each contact is assigned arespective intensity that is a portion of the aggregate intensity. FIG.5D illustrates assigning the aggregate intensity to contacts 552A-552Ebased on their distance from the center of force 554. In this example,each of contacts 552A, 552B, and 552E are assigned an intensity ofcontact of 8 intensity units of the aggregate intensity, and each ofcontacts 552C and 552D are assigned an intensity of contact of 4intensity units of the aggregate intensity. More generally, in someimplementations, each contact j is assigned a respective intensity Ijthat is a portion of the aggregate intensity, A, in accordance with apredefined mathematical function, Ij=A·(Dj/ΣDi), where Dj is thedistance of the respective contact j to the center of force, and ΣDi isthe sum of the distances of all the respective contacts (e.g., i=1 tolast) to the center of force. The operations described with reference toFIGS. 5C-5D can be performed using an electronic device similar oridentical to device 100, 300, or 500. In some embodiments, acharacteristic intensity of a contact is based on one or moreintensities of the contact. In some embodiments, the intensity sensorsare used to determine a single characteristic intensity (e.g., a singlecharacteristic intensity of a single contact). It should be noted thatthe intensity diagrams are not part of a displayed user interface, butare included in FIGS. 5C-5D to aid the reader.

In some embodiments, a portion of a gesture is identified for purposesof determining a characteristic intensity. For example, atouch-sensitive surface optionally receives a continuous swipe contacttransitioning from a start location and reaching an end location, atwhich point the intensity of the contact increases. In this example, thecharacteristic intensity of the contact at the end location is,optionally, based on only a portion of the continuous swipe contact, andnot the entire swipe contact (e.g., only the portion of the swipecontact at the end location). In some embodiments, a smoothing algorithmis, optionally, applied to the intensities of the swipe contact prior todetermining the characteristic intensity of the contact. For example,the smoothing algorithm optionally includes one or more of: anunweighted sliding-average smoothing algorithm, a triangular smoothingalgorithm, a median filter smoothing algorithm, and/or an exponentialsmoothing algorithm. In some circumstances, these smoothing algorithmseliminate narrow spikes or dips in the intensities of the swipe contactfor purposes of determining a characteristic intensity.

The intensity of a contact on the touch-sensitive surface is,optionally, characterized relative to one or more intensity thresholds,such as a contact-detection intensity threshold, a light press intensitythreshold, a deep press intensity threshold, and/or one or more otherintensity thresholds. In some embodiments, the light press intensitythreshold corresponds to an intensity at which the device will performoperations typically associated with clicking a button of a physicalmouse or a trackpad. In some embodiments, the deep press intensitythreshold corresponds to an intensity at which the device will performoperations that are different from operations typically associated withclicking a button of a physical mouse or a trackpad. In someembodiments, when a contact is detected with a characteristic intensitybelow the light press intensity threshold (e.g., and above a nominalcontact-detection intensity threshold below which the contact is nolonger detected), the device will move a focus selector in accordancewith movement of the contact on the touch-sensitive surface withoutperforming an operation associated with the light press intensitythreshold or the deep press intensity threshold. Generally, unlessotherwise stated, these intensity thresholds are consistent betweendifferent sets of user interface figures.

An increase of characteristic intensity of the contact from an intensitybelow the light press intensity threshold to an intensity between thelight press intensity threshold and the deep press intensity thresholdis sometimes referred to as a “light press” input. An increase ofcharacteristic intensity of the contact from an intensity below the deeppress intensity threshold to an intensity above the deep press intensitythreshold is sometimes referred to as a “deep press” input. An increaseof characteristic intensity of the contact from an intensity below thecontact-detection intensity threshold to an intensity between thecontact-detection intensity threshold and the light press intensitythreshold is sometimes referred to as detecting the contact on thetouch-surface. A decrease of characteristic intensity of the contactfrom an intensity above the contact-detection intensity threshold to anintensity below the contact-detection intensity threshold is sometimesreferred to as detecting liftoff of the contact from the touch-surface.In some embodiments, the contact-detection intensity threshold is zero.In some embodiments, the contact-detection intensity threshold isgreater than zero.

In some embodiments described herein, one or more operations areperformed in response to detecting a gesture that includes a respectivepress input or in response to detecting the respective press inputperformed with a respective contact (or a plurality of contacts), wherethe respective press input is detected based at least in part ondetecting an increase in intensity of the contact (or plurality ofcontacts) above a press-input intensity threshold. In some embodiments,the respective operation is performed in response to detecting theincrease in intensity of the respective contact above the press-inputintensity threshold (e.g., a “down stroke” of the respective pressinput). In some embodiments, the press input includes an increase inintensity of the respective contact above the press-input intensitythreshold and a subsequent decrease in intensity of the contact belowthe press-input intensity threshold, and the respective operation isperformed in response to detecting the subsequent decrease in intensityof the respective contact below the press-input threshold (e.g., an “upstroke” of the respective press input).

FIGS. 5E-5H illustrate detection of a gesture that includes a pressinput that corresponds to an increase in intensity of a contact 562 froman intensity below a light press intensity threshold (e.g., “IT_(L)”) inFIG. 5E, to an intensity above a deep press intensity threshold (e.g.,“IT_(D)”) in FIG. 5H. The gesture performed with contact 562 is detectedon touch-sensitive surface 560 while cursor 576 is displayed overapplication icon 572B corresponding to App 2, on a displayed userinterface 570 that includes application icons 572A-572D displayed inpredefined region 574. In some embodiments, the gesture is detected ontouch-sensitive display 504. The intensity sensors detect the intensityof contacts on touch-sensitive surface 560. The device determines thatthe intensity of contact 562 peaked above the deep press intensitythreshold (e.g., “IT_(D)”). Contact 562 is maintained on touch-sensitivesurface 560. In response to the detection of the gesture, and inaccordance with contact 562 having an intensity that goes above the deeppress intensity threshold (e.g., “IT_(D)”) during the gesture,reduced-scale representations 578A-578C (e.g., thumbnails) of recentlyopened documents for App 2 are displayed, as shown in FIGS. 5F-5H. Insome embodiments, the intensity, which is compared to the one or moreintensity thresholds, is the characteristic intensity of a contact. Itshould be noted that the intensity diagram for contact 562 is not partof a displayed user interface, but is included in FIGS. 5E-5H to aid thereader.

In some embodiments, the display of representations 578A-578C includesan animation. For example, representation 578A is initially displayed inproximity of application icon 572B, as shown in FIG. 5F. As theanimation proceeds, representation 578A moves upward and representation578B is displayed in proximity of application icon 572B, as shown inFIG. 5G. Then, representations 578A moves upward, 578B moves upwardtoward representation 578A, and representation 578C is displayed inproximity of application icon 572B, as shown in FIG. 5H. Representations578A-578C form an array above icon 572B. In some embodiments, theanimation progresses in accordance with an intensity of contact 562, asshown in FIGS. 5F-5G, where the representations 578A-578C appear andmove upwards as the intensity of contact 562 increases toward the deeppress intensity threshold (e.g., “IT′D”). In some embodiments, theintensity, on which the progress of the animation is based, is thecharacteristic intensity of the contact. The operations described withreference to FIGS. 5E-5H can be performed using an electronic devicesimilar or identical to device 100, 300, or 500.

In some embodiments, the device employs intensity hysteresis to avoidaccidental inputs sometimes termed “jitter,” where the device defines orselects a hysteresis intensity threshold with a predefined relationshipto the press-input intensity threshold (e.g., the hysteresis intensitythreshold is X intensity units lower than the press-input intensitythreshold or the hysteresis intensity threshold is 75%, 90%, or somereasonable proportion of the press-input intensity threshold). Thus, insome embodiments, the press input includes an increase in intensity ofthe respective contact above the press-input intensity threshold and asubsequent decrease in intensity of the contact below the hysteresisintensity threshold that corresponds to the press-input intensitythreshold, and the respective operation is performed in response todetecting the subsequent decrease in intensity of the respective contactbelow the hysteresis intensity threshold (e.g., an “up stroke” of therespective press input). Similarly, in some embodiments, the press inputis detected only when the device detects an increase in intensity of thecontact from an intensity at or below the hysteresis intensity thresholdto an intensity at or above the press-input intensity threshold and,optionally, a subsequent decrease in intensity of the contact to anintensity at or below the hysteresis intensity, and the respectiveoperation is performed in response to detecting the press input (e.g.,the increase in intensity of the contact or the decrease in intensity ofthe contact, depending on the circumstances).

For ease of explanation, the descriptions of operations performed inresponse to a press input associated with a press-input intensitythreshold or in response to a gesture including the press input are,optionally, triggered in response to detecting either: an increase inintensity of a contact above the press-input intensity threshold, anincrease in intensity of a contact from an intensity below thehysteresis intensity threshold to an intensity above the press-inputintensity threshold, a decrease in intensity of the contact below thepress-input intensity threshold, and/or a decrease in intensity of thecontact below the hysteresis intensity threshold corresponding to thepress-input intensity threshold. Additionally, in examples where anoperation is described as being performed in response to detecting adecrease in intensity of a contact below the press-input intensitythreshold, the operation is, optionally, performed in response todetecting a decrease in intensity of the contact below a hysteresisintensity threshold corresponding to, and lower than, the press-inputintensity threshold.

As used herein, an “installed application” refers to a softwareapplication that has been downloaded onto an electronic device (e.g.,devices 100, 300, and/or 500) and is ready to be launched (e.g., becomeopened) on the device. In some embodiments, a downloaded applicationbecomes an installed application by way of an installation program thatextracts program portions from a downloaded package and integrates theextracted portions with the operating system of the computer system.

As used herein, the terms “open application” or “executing application”refer to a software application with retained state information (e.g.,as part of device/global internal state 157 and/or application internalstate 192). An open or executing application is, optionally, any one ofthe following types of applications:

-   -   an active application, which is currently displayed on a display        screen of the device that the application is being used on;    -   a background application (or background processes), which is not        currently displayed, but one or more processes for the        application are being processed by one or more processors; and    -   a suspended or hibernated application, which is not running, but        has state information that is stored in memory (volatile and        non-volatile, respectively) and that can be used to resume        execution of the application.

As used herein, the term “closed application” refers to softwareapplications without retained state information (e.g., state informationfor closed applications is not stored in a memory of the device).Accordingly, closing an application includes stopping and/or removingapplication processes for the application and removing state informationfor the application from the memory of the device. Generally, opening asecond application while in a first application does not close the firstapplication. When the second application is displayed and the firstapplication ceases to be displayed, the first application becomes abackground application.

Attention is now directed towards embodiments of user interfaces (“UI”)and associated processes that are implemented on an electronic device,such as portable multifunction device 100, device 300, or device 500.

FIGS. 6A-6AF illustrate exemplary user interfaces for initiating andparticipating in a live communication session (e.g., from a messagingapplication), in accordance with some embodiments. The user interfacesin these figures are used to illustrate the processes described below,including the processes in FIG. 7A-7F.

FIG. 6A illustrates device 600 with touch-sensitive display 602. In someembodiments, device 600 includes one or more features of devices 100,300, and 500. In some embodiments, device 600 also includes camera 603,which, at a minimum, includes an image sensor that is capable ofcapturing data representing a portion of the light spectrum (e.g.,visible light, infrared light, or ultraviolet light). In someembodiments, camera 603 includes multiple image sensors and/or othertypes of sensors. In addition to capturing data representing sensedlight, in some embodiments, camera 603 is capable of capturing othertypes of data, such as depth data. For example, in some embodiments,camera 603 also captures depth data using techniques based on speckle,time-of-flight, parallax, or focus. Image data that device 600 capturesusing camera 603 includes data corresponding to a portion of the lightspectrum for a scene within the field of view of the camera.Additionally, in some embodiments, the captured image data also includesdepth data for the light data. In some other embodiments, the capturedimage data contains data sufficient to determine or generate depth datafor the data for the portion of the light spectrum.

In some examples, electronic device 600 includes a depth camera (e.g.,as part of camera 603), such as an infrared camera, a thermographiccamera, or a combination thereof. In some examples, the device furtherincludes a light-emitting device (e.g., light projector), such an IRflood light, a structured light projector, or a combination thereof. Thelight-emitting device is, optionally, used to illuminate the subjectduring capture of the image by a visible light camera and a depth camera(e.g., an IR camera) and the information from the depth camera and thevisible light camera are used to determine a depth map of differentportions of subject captured by the visible light camera. In someembodiments, a depth map (e.g., depth map image) contains information(e.g., values) that relates to the distance of objects in a scene from aviewpoint (e.g., a camera). In one embodiment of a depth map, each depthpixel defines the position in the viewpoint's Z-axis where itscorresponding two-dimensional pixel is located. In some examples, adepth map is composed of pixels wherein each pixel is defined by a value(e.g., 0-255). For example, the “0” value represents pixels that arelocated at the most distant place in a “three dimensional” scene and the“255” value represents pixels that are located closest to a viewpoint(e.g., camera) in the “three dimensional” scene. In other examples, adepth map represents the distance between an object in a scene and theplane of the viewpoint.) In some embodiments, the depth map includesinformation about the relative depth of various features of an object ofinterest in view of the depth camera (e.g., the relative depth of eyes,nose, mouth, ears of a user's face). In some embodiments, the depth mapincludes information that enables the device to determine contours ofthe object of interest in a z direction. In some embodiments, thelighting effects described herein are displayed using disparityinformation from two cameras (e.g., two visual light cameras) for rearfacing images and using depth information from a depth camera combinedwith image data from a visual light camera for front facing images(e.g., selfie images). In some embodiments, the same user interface isused when the two visual light cameras are used to determine the depthinformation and when the depth camera is used to determine the depthinformation, providing the user with a consistent experience, even whenusing dramatically different technologies to determine the informationthat is used when generating the lighting effects. In some embodiments,while displaying the camera user interface with one of the lightingeffects applied, the device detects selection of a camera switchingaffordance and switches from the front facing cameras (e.g., a depthcamera and a visible light camera) to the rear facing cameras (e.g., twovisible light cameras that are spaced apart from each other) (or viceversa) while maintaining display of the user interface controls forapplying the lighting effect and replacing display of the field of viewof the front facing cameras to the field of view of the rear facingcameras (or vice versa).

As illustrated in FIG. 6A, device 600 displays messaging user interface604 of a messaging application. Messaging user interface 604 includesmessage region 606 that includes messages 608A-608D in a messageconversation between a group of participants including three or moreparticipants (“The Dream Team”).

While displaying messaging user interface 604, device 600 receives dataindicating that a live communication session is active (e.g., aninvitation to join a live communication session), where the livecommunication session is available to the participants in the group TheDream Team. In some embodiments, the live communication session is anaudio communication session or an audio/video communication session.

In response to receiving the data indicating that the live communicationsession is active, device 600 displays a notification indicating thatthe live communication session is active. As illustrated in FIG. 6B, thenotification includes notification 614 as a message from participantPablo in message region 606, indicating that Pablo initiated the livecommunication session. Notification 614 indicates the type of livecommunication session with text “VIDEO” and a video camera icon.Notification 614 also indicates the number of participants currentlyactive in the live communication session (“1 PERSON ACTIVE”). In someembodiments, notification 614 indicates the name of one or more of theparticipants that are currently active in the live communicationsession. Notification 614 also includes live communication joinaffordance 614A, which when selected, causes device 600 to initiate aprocess for joining the live communication session represented bynotification 614. Techniques for joining the live communication sessionare described in greater detail below.

Optionally, in response to receiving the data indicating that the livecommunication session is active, device 600 generates an audio outputindicating that the live communication session is active. In someembodiments, the audio output is different than an audio outputassociated with a messaging application (e.g., a notification of areceived message) and an audio output associated with a phoneapplication (e.g., a ring for an incoming call). In some embodiments,the audio output is a hybrid of audio associated with a messagenotification and audio associated with an incoming call notification. Insome embodiments, the audio output shares audio characteristics (e.g.,common tones, notes, cadence) with both the audio outputs for the phonecall and message notifications.

Also in response to receiving the data indicating that the livecommunication session is active, device 600 displays live communicationaffordance 610. In some embodiments, live communication affordance 610is displayed before the live communication session is initiated (e.g.,in a greyed out state) and is visually emphasized (e.g., highlighted) ormodified (e.g., to a non-greyed out state) when the live communicationsession is initiated. In some embodiments, selection of livecommunication affordance 610 or options affordance 612 initiates aprocess for joining the live communication session. In some embodiments,notification 614 and/or live communication affordance 610 are displayedwithout audio and/or haptic output (e.g., phone ring output) (e.g.,because it is being displayed in an active message conversation that theuser is likely already focused on).

As illustrated in FIG. 6C, as additional messages are added to themessage conversation in message region 606, notification 614 moves up toa different location in message region 606. In some embodiments, device600 moves notification 614 to a location outside the message region(e.g., notification 614 transitions from a message to a banner at thetop of display 602 outside message region 606).

In some embodiments, device 600 moves notification 614 in response to anotification relocation criteria being met. In some embodiments, thenotification relocation criteria includes one or more criterion selectedfrom the group consisting of: a predetermined time elapsing (e.g.,elapsing after notification 614 is displayed), messaging user interface604 ceasing to be displayed, the messaging application being closed,and/or the entirety of notification 614 or a portion of notification 614being hidden in message region 606 (e.g., due to new incoming messagespopulating message region 606). In some embodiments, notification 614 ismoved to the bottom, right, or left side of display 602. In someembodiments, notification 614 is a banner notification displayed nearthe top of display 612 directly in response to receiving the dataindicating that the live communication session is available.

In some embodiments, device 600 receives reminder data indicating areminder to join the live communication session (e.g., a reminder sentby a participant in the conversation or after a predetermined period oftime). As illustrated in FIG. 6D, in response to receiving dataindicating a reminder to join the live communication session, device 600displays full screen notification 616 and outputs audio output 618. Insome embodiments, audio output 618 is different than an audio outputassociated with a messaging application (e.g., a notification of areceived message) and an audio output associated with a phoneapplication (e.g., a ring for an incoming call). In some embodiments,audio output 618 is a hybrid of audio associated with a messagenotification and audio associated with an incoming call notification. Insome embodiments, audio output 618 is shorter than an audio outputassociated with a phone application (e.g., a ring for an incoming call)and is longer than an audio output associated with a messagingapplication (e.g., a notification of a received message). In someembodiments, the ring for an incoming call includes multiple repetitionsof a respective audio output and audio output 618 is a single repetitionof the respective audio output. In some embodiments, audio output 618shares audio characteristics (e.g., common tones, notes, cadence) withboth the audio outputs for the phone call and message notifications. Insome embodiments, device 600 outputs a haptic output. As illustrated inFIG. 6D, full screen notification 616 includes the name of the group(“THE DREAM TEAM”), names of participants currently active in the livecommunication session (“PABLO & STEPHEN”), and the name of theparticipant who originally initiated the live communication session orwho initiated the reminder (“PABLO”). In some embodiments, full screennotification 616 includes names of participants that have been invitedto join the live communication session (e.g., according to the order inwhich they were added to the group). Full screen notification 616 alsoincludes answer affordance 620 for joining the live communicationsession (e.g., to immediately join the live communication session ordisplay a menu with options for joining the live communication session),“Not Now” affordance 622 for dismissing full screen notification 616,and message affordance 624 for returning to messaging user interface 604or sending a message to the Dream Team group. Full screen notificationalso includes an image (e.g., video of a user of device 600 from camera603) (e.g., as a preview of the image of the user that would be seen byother participants in the live communication session, once joined).

As illustrated in FIG. 6D, device 600 receives (e.g., detects) userinput 650A (e.g., a tap) corresponding to selection of “Not Now”affordance 622. As illustrated in FIG. 6E, in response to user input650A, device 600 ceases to display full screen notification 616 andreturns to messaging user interface 604.

As illustrated in FIG. 6E, device 600 receives (e.g., detects) userinput 650B (e.g., a tap) corresponding to selection of optionsaffordance 612. In response to receiving user input 650B, device 600expands the header of messaging user interface 604 to include audioaffordance 626A, video join affordance 626B, and group detailsaffordance 626C.

As illustrated in FIG. 6F, device 600 receives (e.g., detects) userinput 650C (e.g., a tap) corresponding to selection of video joinaffordance 626B. In some embodiments, device 600 joins the livecommunication session directly in response to receiving user input 650B(e.g., without requiring additional inputs). In some embodiments, device600 joins the live communication session and transmits both video andaudio. In some embodiments, device 600 joins the live communicationsession and transmits only audio (e.g., without transmitting video). Insome embodiments, in response to selection of audio affordance 626A,device 600 joins the live communication session with audio only orinitiates a separate phone conference call with the participants of thegroup.

As illustrated in FIG. 6G, in response to receiving user input 650C,device 600 displays live communication interface 628 of a livecommunication application. Live communication interface 628 includes anindication of the group (“THE DREAM TEAM”), an image (e.g., video of auser of device 600 from a camera on the front of device 600 (e.g.,camera 603)), an indication of the participants that are currentlyactive in the live communication session (“PABLO & STEPHEN”), anindication of the participant that initiated the live communicationsession (“VIDEO CALL FROM PABLO”), and call control menu 630. Callcontrol menu 630 includes handle affordance 631 effects affordance 632,call affordance 634, and menu affordance 636, and.

As illustrated in FIG. 6G, device 600 receives (e.g., detects) userinput 650D on menu affordance 636. As illustrated in FIG. 6H, inresponse to user input 650D, device 600 expands call control menu 630 todisplay additional information and controls related to the livecommunication session. In some embodiments, device 600 expands callcontrol menu 630 in response to an upward swipe starting near handleaffordance 631. When expanded, call control menu 630 further includesaudio ON/OFF affordance 638, audio source menu affordance 640, videoON/OFF affordance 642, group name 644 (with an indication of the numberof participants in the group), group message affordance 646, list 648including affordances 648A-648E corresponding to participants of thegroup, and add participant affordance 652. In some embodiments, inresponse to receiving selection of group message affordance 646, device600 launches and/or displays messaging user interface 604 (e.g., FIG.6F) for sending a message to the participants of the group.

As illustrated in FIG. 6H, the affordances of participants in list 648include an indication of the communication status of the respectiveparticipant with respect to the live communication session.Representation 648A indicates that participant Pablo is connected to thelive communication session and is providing video data (e.g., a livemedia stream including video and audio data). Representation 648Bindicates that participant Stephen is connected and providing audio data(e.g., no video). Representations 648C and 648D indicate thatparticipants Marcel and Allison, respectively, are not connected to thelive communication session (e.g., not providing video and audio data).Participants that are not connected have either never joined the livecommunication session or have previously joined and then left the livecommunication session. Exemplary participant communication status typesinclude audio-only (e.g., participant is communication using audioonly); video (e.g., participant is talking using video and audio); videopaused (e.g., participant's video is paused); video not decoded (e.g.,participants video stream cannot be processed because of latency issues,issues with the format of the video stream, etc.); left (e.g.,participant has left the live communication session); and waiting tojoin (e.g., participant was invited to the live communication sessionbut has not joined the live communication session yet).

In some embodiments, participants of the live video communicationinclude participants of the group that are currently connected to thecommunication session and participants of the group that are notcurrently connected to the live video communication session. That is,all participants of the group are referred to as participants of thelive communication session, regardless of connection status. When a livecommunication session is active, participants can join (connect to) orleave (disconnect from) the live communication session.

As illustrated in FIG. 6I, device 600 receives (e.g., detects) userinput 650E (e.g., a tap) corresponding to selection of video ON/OFFaffordance 642. As illustrated in FIG. 6J, in response to receiving userinput 650E, device 600 modifies video ON/OFF affordance 642 to indicatea video OFF state and modifies call affordance 634 to indicate an audioonly state by changing the representation of a video camera in FIG. 6Ito a representation of a phone in FIG. 6J. In an audio only state,device 600 joins the live communication session with audio only (e.g.,device 600 does not provide video data).

As illustrated in FIG. 6J, device 600 receives (e.g., detects) userinput 650F (e.g., a tap) corresponding to selection of call affordance634. As illustrated in FIG. 6K, in response to receiving user input650F, device 600 initiates a connection to join the live communicationsession in an audio only mode. As illustrated in FIG. 6K, device 600ceases displaying expanded call control menu 630 (e.g., device 600reduces call control menu 630) and displays a connecting screen 654. Insome embodiments, device 600 initiates a connection to join the livecommunication session and displays a connecting screen 654 in responseto user input 650C on video join affordance 626B. In some embodiments,notification 614 includes an affordance (e.g., the “JOIN” icon onnotification 614 or entire notification 614 itself is selectable). Insome embodiments, device 600 initiates a connection to join the livecommunication session and displays a connecting screen 654 in responseto selection of notification 614.

As illustrated in FIG. 6L, after device 600 has joined the livecommunication session, device 600 displays live communication userinterface 656 including a representation 658 of the user of device 600,a representation 670A of participant Pablo (e.g., the participant thatinitiated the live communication session) and representation 670B ofparticipant Stephen. In accordance with the communication statusindicated in list 648, representation 670A of participant Pablo includesa live video stream. Since participant Stephen is providing only audiodata, representation 670B of participant Pablo includes an avatar. Insome embodiments, the layout of the representations on livecommunication user interface 656 is based on the number of participantscurrently connected to the live communication session (e.g., twoparticipants and the user of device 600). The layout illustrated in FIG.6L is referred to as a 2-on-1 layout, with a first positioncorresponding to the position of representation 670A and a secondposition corresponding to the position of representation 670B.

In some embodiments, when device 600 joins the live communicationsession, device 600 determines whether the number of participants (e.g.,either active or total (active and invited)) exceeds a predeterminedthreshold number. In accordance with a determination that the number ofparticipants exceeds the threshold, device 600 displays an overflowregion with overflow representations (e.g., thumbnails) as described ingreater detail below. Alternatively, in accordance with a determinationthat the number of participants does not exceed the threshold, device600 does not display an overflow region with overflow representations.

When device 600 has joined a live communication session, device 600transmits a live media stream to the other participants (e.g., theparticipants that have joined the live communication session). When aparticipant associated with another device has joined the livecommunication session and device 600 has also joined, device 600receives a live media stream corresponding to that participant.

As illustrated in FIG. 6L, while displaying live communication userinterface 656, device 600 receives (e.g., detects) user input 650G(e.g., a tap) on display 602. As illustrated in FIG. 6M, in response toreceiving user input 650G, device 600 displays call control menu 630.Representations 670A and 670B are reduced in size and representations670B and 658 are moved upward on display 602 such that call control menu630 does not overlap with the representations. In contrast to theconfiguration illustrated in FIG. 6L where representations 670A and 670Bare vertically aligned and do not overlap, when call control menu 630 isdisplayed in FIG. 6M, representations 670A and 670B are offset from eachother and overlap.

As illustrated in FIG. 6N, device 600 receives (e.g., detects) userinput 650H (e.g., a tap) on menu affordance 636 in call control menu630. As illustrated in FIG. 6O, in response to receiving user input650H, device 600 expands call control menu 630 (e.g., as discussed withrespect to FIG. 6H). In some embodiments, device 600 displays reminderaffordances in call control menu 630 at representations of participantswho are not connected to the live communication session. As illustratedin FIG. 6O, list 648 indicates that participants Marcel and Allison arenot connected to the live communication session. Since participantsMarcel and Allison (e.g., devices associated with participants Marceland Allison) are not connected to the live communication session,expanded call control menu 630 includes ring affordances 662 and 664 atrepresentations 648C and 648D of participants Marcel and Allison,respectively, in list 648. In response to selection of ring affordance662 or 664, device 600 causes a notification (e.g., audio output, suchas a call or a ring) to be provided at a device associated with theparticipant corresponding to the selected ring affordance. In someembodiments, the notification is a new notification that is differentthan a previous notification sent to the participant. In someembodiments, the new notification is more intrusive than the previousnotification (e.g., an audio ring, haptic output, or full screennotification, as compared to a banner notification without audio orhaptic output). In this way, a user can remind a participant who has notjoined the live communication session that the live communicationsession is active and available for the participant to join (e.g.,similar to the way in which the user of device 600 is reminded by thefull screen notification illustrated in FIG. 6D).

As illustrated in FIG. 6O, device 600 receives (e.g., detects) userinput 650I (e.g., a tap) corresponding to selection of video ON/OFFaffordance 642. In response to receiving user input 650I, device 600provides a live video stream (e.g., from a camera on device 600) to thelive communication session and modifies video ON/OFF affordance 642 toindicate that video is ON, as illustrated in FIG. 6P. In FIG. 6Q, device600 receives (e.g., detects) user input 650J (e.g., a down swipe) ondisplay 602 to reduce call control menu 630. As illustrated in FIG. 6R,when call control menu 630 is reduced, representation 658 correspondingto the user of device 600 includes video from camera 603 of device 600.

As illustrated in FIG. 6S, device 600 receives (e.g., detects) userinput 650K (e.g., a tap) corresponding to selection of call affordance634. In response to user input 650K, device 600 disconnects from thelive communication session. FIG. 6T illustrates an embodiment ofmessaging user interface 604 of the messaging application displayedafter device 600 has disconnected from the live communication session.Although device 600 has disconnected from the live communicationsession, the live communication session remains active. In someembodiments, a live communication session remains active as long as atleast one of the participants is connected, even if the participant thatinitiated the live video communication session leaves the session. Asillustrated in FIG. 6T, device 600 maintains display of notification 614and live communication affordance 610, indicating that the livecommunication session is active. Notification 614 is updated to indicatethe duration of the live communication session (e.g., three minutes andforty-two seconds, corresponding to the amount of time since the livecommunication session was initiated).

While displaying notification 614 indicating that the live communicationsession is active, device 600 receives data indicating that the livecommunication session is no longer active (e.g., has ended). In someembodiments, the live communication session ends when no participant isconnected to the live communication session (e.g., all participants thathad either initiated or joined the live communication session are nolonger connected). In response to receiving the data indicating that thelive communication session is no longer active, device 600 ceasesdisplaying notification 614 and/or updates notification 614 to indicatethat the live communication session is no longer active. FIG. 6Uillustrates an embodiment of messaging user interface 604 of themessaging application displayed after the live communication session hasended (e.g., in response to receiving the data indicating that the livecommunication session is no longer active). As illustrated in FIG. 6U,notification 614 is updated to include an indication (e.g., text “CALLENDED”) that the live communication session is no longer active and livecommunication affordance 610 is removed. In some embodiments, livecommunication affordance 610 and/or notification 614 is greyed out toindicate that the live communication session is no longer active. Insome embodiments, notification 614 is no longer selectable after thelive communication session has ended. In some embodiments, after thelive communication session has ended, notification 614 remainsselectable and can be selected to initiate a new live communicationsession with the group. In some embodiments, if device 600 had notjoined the live communication session, device 600 updates notification614 to indicate that the user missed the live communication session(e.g., “MISSED CALL”).

As illustrated in FIG. 6U, device 600 receives (e.g., detects) userinput 650L (e.g., a tap) corresponding to selection of back affordance611. As illustrated in FIG. 6V, in response to receiving user input650L, device 600 displays user interface 671 of the messagingapplication, which includes list 673 of affordances 673A-673G foraccessing respective message conversations. Affordance 673D correspondsto a message conversation between a group of three participants(Matthew, Danielle, and the user of device 600). Affordance 673Dincludes live communication indicator 675, which indicates that there isan active live communication session for this group. In contrast, sincea live communication session for The Dream Team is not active,affordance 673A corresponding to The Dream Team does not include livecommunication indicator 675.

Turning now to FIG. 6W, device 600 is illustrated as displaying aninterface other than messaging user interface 604. In FIG. 6W, device600 displays lock screen 668 indicating that device 600 is in a lockedstate. Lock screen 668 includes notifications 677A and 677B.Notification 677A indicates that a message from a user John Appleseedwas received five minutes ago. Notification 677B represents a livecommunication session that is no longer available to join (e.g., amissed video call). Missed video call notification includes indicationsof the invited participants (John, Amy, & Rodrigo), the status of thelive communication session (“MISSED”), and a time when the livecommunication session was either initiated or ended (“1 HR AGO”). In thelock state, device 600 does not display an active application.

While in the locked state, device 600 receives data indicating that alive communication session is available to join. In some embodiments,the response of device 600 to receiving the data indicating that a livecommunication session is available to join depends on a context ofdevice 600, the available live communication session, or otherfactor(s). In some embodiments, in accordance with the livecommunication session being available between two participants (e.g., aone-on-one communication session), device 600 displays a first type ofnotification. As illustrated in FIG. 6X, the first type of notificationincludes full screen notification 672 and audio output 674. In someembodiments, audio output 674 is different than an audio outputassociated with a phone call notification and an audio output associatedwith a message notification. In some embodiments, audio output 674shares audio characteristics (e.g., common tones, notes, cadence) withboth the audio outputs for the phone call and message notifications.Alternatively, in accordance with the live communication session beingavailable between three or more participants (e.g., a group livecommunication session), device 600 displays a second type ofnotification. As illustrated in FIG. 6Y, the second type of notificationincludes a banner notification 676 (e.g., not full screen) and hapticoutput 678 (e.g., but no audio output). In some embodiments, inaccordance with the live communication session being available betweenthree or more participants (e.g., a group live communication session),device 600 initially displays a full screen notification (e.g., similarto full screen notification 616 illustrated in FIG. 6E) and thendisplays (e.g., transitions to) banner notification 676. In someembodiments, notification 676 includes an affordance that when selectedinitiates a process for joining the corresponding live communicationsession (e.g., as described above with reference to FIGS. 6F-6L).

In contrast to notification 677B, which corresponds to a missed videocall, notification 676 corresponds to an active live communicationsession. Notification 676 includes an indication of the group (The DreamTeam), the participant that performed an action triggering presentationof the notification (Pablo), and a time when the live communicationsession was initiated (“NOW”). In some embodiments, notification 676updates to accurately reflect the time at which the live communicationsession was initiated and the status of the live communication session.In some embodiments, when the live communication session represented bynotification 676 ends, the text of notification 676 is updated from“JOIN THIS VIDEO CALL FROM PABLO” to “MISSED VIDEO CALL FROM PABLO”.

In some embodiments, in accordance with a determination that an activeapplication is not being displayed (as in FIG. 6W), device 600 displaysa full screen notification in response to receiving the data indicatingthat a live communication session is available to join. In someembodiments, in accordance with a determination that an activeapplication (e.g., the messaging application) is being displayed, device600 displays a non-full screen notification (e.g., a bannernotification).

In some embodiments, in accordance with a determination to alert a userof device 600, device 600 outputs an alert (e.g., a notification or anaudio or haptic output); and in accordance with a determination to notalert the user of device 600, device 600 refrains from outputting thealert. In some embodiments, the determination to alert a user is basedon a disturbance criterion (e.g., whether the device is locked orunlocked). In some embodiments, in accordance with device 600 beinglocked (e.g., as illustrated in FIG. 6Y), device outputs an alert (e.g.,haptic output 678); and in accordance with device 600 being unlocked(e.g., as illustrated in FIG. 6Z), device 600 displays a notification(e.g., banner notification 676 and refrains from providing the alert(e.g., no haptic output).

In some embodiments, the disturbance criterion indicates whether a useris looking at device 600 or not (e.g., based on data from a sensor onthe front of device 600). FIG. 6AA illustrates user 684 holding device600. The user's gaze 680 is not directed at device 600. Device 600determines whether user 684 is not looking at device 600, and inaccordance with a determination that user 684 is not looking at device600, device 600 provides a non-visual notification 682 (e.g., a hapticor audio output). As illustrated in FIG. 6AB, user's gaze 680 isdirected at device 600. In accordance with a determination that user 684is looking at device 600, device 600 refrains from providing anon-visual notification (e.g., device 600 displays a banner notificationwithout haptic or audio output). In some embodiments, device 600determines whether a user is looking at device 600 using an image sensorconfigured to identify a face of a user.

Turning to FIGS. 6AC-6AF, device 600A includes display 602A (e.g.,touch-sensitive display), rotatable input mechanism 601A, and mechanicalbutton 601B. In some embodiments, device 600A includes features ofdevice 100, 300, 500, or 600. In some embodiments, device 600Acommunicates with device 600 (e.g., via two-way wireless communication).In some embodiments, device 600A is in a paired relationship with device600.

In some embodiments, device 600A receives (e.g., from device 600) dataindicating that a live communication session that is available to thethree or more participants is active. As illustrated in FIG. 6AC, inresponse to receiving the data indicating that the live communicationsession is active, device 600A displays user interface 604A, whichincludes notification 605, live communication join affordance 607, anddismiss affordance 609. In some embodiments, notification 605 includesfeatures of notification 614 described above (e.g., participant names,number of participants, the name of the participant that triggered thenotification). In some embodiments, device 600A generates audio and/orhaptic output in response to receiving the data indicating that the livecommunication session is active. In some embodiments, device 600Agenerates audio and/or haptic output different from audio and/or hapticoutput generated by device 600 in response to receiving the dataindicating that the live communication session is active. In someembodiments, a notification (e.g., display, audio, and/or haptic output)that a live communication session is active is provided at device 600(e.g., not at device 600A) in response to receiving initial dataindicating that a live communication session is active, and anotification is provided at both device 600 and device 600A in responseto receiving reminder data indicating a reminder that the livecommunication session is active. In some embodiments, a notification(e.g., display, audio, and/or haptic output) that a live communicationsession is active is provided at both device 600 and device 600A inresponse to receiving initial data indicating that a live communicationsession is active, and a notification is provided at device 600 (e.g.,not at device 600A) in response to receiving reminder data indicating areminder that the live communication session is active.

In some embodiments, device 600A receives user input corresponding toselection of live communication join affordance 607 (e.g., a tap on livecommunication affordance 617). In response to receiving selection oflive communication join affordance 607, device 600A causes device 600and/or device 600A to join the live communication session or initiate aprocess for joining the live communication session. In some embodiments,In response to receiving selection of live communication join affordance607, device 600A causes device 600 to display live communication userinterface 628 (FIG. 6G), 654 (FIG. 6K), or 656 (FIG. 6L).

FIG. 6AD illustrates user interface 604A displayed on device 600A afterjoining the live communication session. In some embodiments, device 600Aincludes a microphone for a user to provide audio inputs to livecommunication session via device 600A. In FIG. 6AD, user interface 604Aincludes volume control 616, indication 613 of the participants (“PABLO& 2 OTHERS”) and duration (“9:45”) of the live communication session,end call affordance 615, audio ON/OFF affordance 617, and menuaffordance 619.

In FIG. 6AE, device 600A receives (e.g., detects) user input 650M (e.g.,upward swipe gesture beginning at or near menu affordance 619). Asillustrated in FIG. 6AF, in response to receiving user input 650M,device 600A modifies indication 613 (“PARTY OF 4”) and displays keypadaffordance 621, audio source affordance 623 (e.g., to participate in thelive communication session using device 600), and list 625 ofparticipants of the live communication session (e.g., joined and/orinvited participants). In some embodiments, device 604A displays list625 in response to receiving user input 650M on user interface 604 inFIG. 6AC (e.g., prior to joining the live communication session). Insome embodiments, list 625 includes an indication of the communicationstatus of the participants. In some embodiments, list 625 includesreminder affordances that can be selected to send reminder notificationsto participants that have not joined the live communication session.

FIGS. 7A-7F are a flow diagram illustrating a method in accordance withsome embodiments. Method 700 is performed at a device (e.g., 100, 300,500, or 600) with a display. Some operations in method 700 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 700 provides, among other things, anintuitive way for providing an interactive notification (e.g., 614, 672,676) for a live communication session between multiple participants. Themethod reduces the cognitive burden on a user using innovativetechniques to provide a notification for a live communication sessionbetween multiple participants, thereby creating a more efficienthuman-machine interface. For battery-operated computing devices,enabling a user to quickly interact with the notification for livecommunication sessions between multiple participants is faster and moreefficient, and thus, conserving power and increasing the time betweenbattery charges.

At block 702, the device displays a messaging user interface (e.g., 604)of a messaging application. The messaging user interface (e.g., 604)includes a message region (e.g., 606). The message region includes aplurality of messages (e.g., 608A-608D) in a message conversationbetween three or more participants.

At block 704, the device receives first data indicating that a livecommunication session (e.g., an audio communication session or anaudio/video communication session) that is available to the three ormore participants is active. In some embodiments, a participant of alive communication session has a status of “currently joined” (e.g.,providing a live media stream to the other participants; also referredto as “connected” or “active”) or “not currently joined” (e.g.,participants that have never joined the session or have previouslyjoined the session and then left the session; also referred to as “notconnected” or “invited”). In some embodiments, participants of the livecommunication session include participants that are currently joined inthe live communication session and participants that are not currentlyjoined in the live communication session (e.g., 648A-648D). When a livecommunication session is active, participants join (connect to) or leave(disconnect from) the live communication session. When a participantassociated with another device has joined the live communicationsession, the device receives a live media stream corresponding to thatparticipant. When the user of the device has joined a live communicationsession, the device transmits a live media stream to the otherparticipants (e.g., the participants that have joined the livecommunication session). In some embodiments, a live communicationsession remains active as long as at least one of the participants isconnected (e.g., even if the participant that initiated the livecommunication session leaves the session).

At block 706, in response to receiving the first data indicating thatthe live communication session between the three or more participants isactive, the device displays a notification (e.g., 614) indicating thatthe live communication session is active. In some embodiments, thenotification indicating that the live communication session is activeincludes a live communication join affordance (e.g., 614A). In someembodiments, the notification includes an indication of the number ofparticipants in the live communication session (e.g., number ofparticipants active in the session; number of participants invited tothe session). In some embodiments, the notification includes anindication (e.g., a name, initials, photo, or avatar) of the participantthat performed an action triggering presentation of the notification onthe display (e.g., a participant that initiated the live communicationsession, or a participant that selected a “ring again” button or thelike to remind the user of the device to join the live communicationsession). In some embodiments, in response to receiving the first dataindicating that the live communication session between the three or moreparticipants is active, the device generates, via one or more audiooutput devices of the device, an audio output (e.g., 618) indicatingthat the live communication session is active, where the audio outputindicating that the live communication session is active is differentthan an audio output associated with a phone call notification and anaudio output associated with a message notification. In someembodiments, the audio output indicating that the live communicationsession is active shares audio characteristics (e.g., common tones,notes, cadence) with both the audio outputs for the phone call andmessage notifications. Displaying a notification (e.g., 614) and/orgenerating an audio output (e.g., 618) indicating that the livecommunication session is active provides the user with feedback aboutthe current state of a live communication session and provides visualfeedback to the user indicating that certain operations associated withthe notification and/or the live communication session will be performedif the user activates the notification (e.g., when the notificationincludes a join affordance). Providing improved visual feedback to theuser enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at block 704, the device displays the notificationindicating that the live communication session is active at a firstlocation in the message region (e.g., 606). In some embodiments, thenotification (e.g., 614) indicating that the live communication sessionis active includes a live communication join affordance (e.g., 614A).

At block 710, while displaying the notification (e.g., 614) indicatingthat the live communication session is active, the device receivessecond data indicating that the live communication session between thethree or more participants is no longer active.

Optionally, at block 712, the device moves the notification indicatingthat the live communication session is active from the first location(e.g., bottom of message region 606) to a second location (e.g., middleor top of message region 606 or top, bottom, right, and/or left sides ofthe display (602)). In some embodiments, moving the notification (e.g.,614) occurs in response to notification relocation criteria being met.In some embodiments, the notification relocation criteria includes oneor more criteria, such as a predetermined amount of time being elapsed(e.g., elapsing after the notification is displayed), the messaging userinterface (e.g. 604) ceasing to be displayed, the messaging applicationbeing closed, the entire notification (e.g., 614) or a portion of thenotification (e.g., 614) being hidden in the message region (e.g., 606)due to new incoming messages populating the message region (e.g., 606),or the like. Moving the notification indicating that the livecommunication session is active provides the user with feedback aboutthe current state of a live communication session, provides visualfeedback to the user indicating that certain operations associated withthe notification and/or the live communication session will beperformed, if selected, and reduces the number of inputs needed toperform an operation. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In addition, reducing the number of inputs needed toperform an operation further enhances the operability of the device andmakes the user-device interface more efficient (e.g., by providing moreaccessible inputs to the user and reducing the number of inputs), whichadditionally, reduces power usage and improves battery life of thedevice by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 714 and 716, the device receives a first userinput (e.g., 650C) corresponding to selection of the live communicationjoin affordance (e.g., 614A or 626B). In response to receiving the firstuser input corresponding to selection of the live communication joinaffordance, the device joins the live communication session (e.g.,joining the live communication session and transmitting both video andaudio from the electronic device and/or joining the live communicationsession and transmitting only audio (without transmitting video) fromthe electronic device). Joining the live communication in response toreceiving a selection of a live communication join affordance that isincluded in a notification reduces the number of inputs needed toperform the operation of joining the live communication session.Reducing the number of inputs needed to perform an operation furtherenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by providing more accessible inputs tothe user and reducing the number of inputs required to join a livecommunication session), which additionally, reduces power usage andimproves battery life of the device by enabling the user to use thedevice more quickly and efficiently.

Optionally, at blocks 718 and 720, the device receives reminder dataindicating a reminder to join the live communication session (e.g., FIG.6D) (e.g., a reminder sent by a participant in the conversation or aftera predetermined period of time). In response to receiving the reminderdata indicating the reminder to join the live communication (e.g., thereminder is sent by a participant in the conversation), the devicegenerates, via one or more audio output devices of the device, an audiooutput (e.g., 618) (e.g., phone ring output) indicating that the livecommunication session is active. In some embodiments, the audio output(e.g., 618) is generated in conjunction with re-displaying anotification (e.g., 616). In some embodiments, the display of thenotification is ongoing at the time the reminder data is received. Insome embodiments, the audio output indicating that the livecommunication session is active is different than an audio outputassociated with a phone call notification and an audio output associatedwith a message notification. In some embodiments, the audio outputindicating that the live communication session is active shares audiocharacteristics (e.g., common tones, notes, cadence) with both the audiooutputs for the phone call and message notifications. In someembodiments, an initial audio output indicating that the livecommunication session is active is different from (e.g., shorter,quieter or less intense than) a reminder audio output indicating thatthe live communication session is active (e.g., the audio outputgenerated based on a participant selecting the “ring” affordance). Insome embodiments, an initial audio output indicating that the livecommunication session is active is a single repetition of a repeatedaudio output that is used for phone calls and a reminder audio outputindicating that the live communication session is active is multiplerepetitions of the repeated audio output that is used for phone calls oris the same as the audio output that is used for phone calls. In someembodiments, in response to receiving the reminder data indicating thereminder to join the live communication, the device displays anindication (e.g., a name, initials, photo, or avatar) of a participantin the live communication session that initiated the reminder.Generating a reminder that includes an audio output when prescribedconditions are met allows the user to quickly recognize that a livecommunication session is available if they missed a visual notification(e.g., 614). Performing an optimized operation when a set of conditionshas been met without requiring further user input enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 722 and 724, in embodiments where the notificationincludes a menu accessing affordance (e.g., 626B), the device receives asecond user input (e.g., 650C) corresponding to selection of the menuaccessing affordance. The device, in response to receiving the seconduser input corresponding to selection of the menu accessing affordance,displays a menu (e.g., 630) that includes a video joining option (e.g.,634) (e.g., affordance, dropdown menu, checkbox) and an audio-onlyoption (e.g., 642) (e.g., affordance, dropdown, checkbox). Displaying amenu that includes different options for joining the live communicationsession in response to receiving a selection of a menu accessingaffordance provides additional control options without cluttering the UIwith additional displayed controls. Providing additional control optionswithout cluttering the UI with additional displayed controls enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 726 and 728, the device, while displaying the menu(e.g., 630), receives a third user input. In accordance with the thirduser input (e.g., 650F) corresponding to selection of the video joiningoption (e.g., 634), the device joins the live communication sessionusing audio and video transmitted from the electronic device. In someembodiments, the device prompts the user to check to see if they want tojoin with audio-only and/or video only (e.g., “do you want to join withvideo?”). Joining the live communication session using audio and videotransmitted from the electronic device when prescribed conditions aremet (e.g., in response to a video joining option being selected) allowsa user to join live communication sessions using audio and video withoutthe user having to select or modify additional input and outputs of thedevice (e.g., turning on camera and/or turning on microphone).Performing an optimized operation when a set of conditions has been metwithout requiring further user input enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 730, in accordance with the third user inputcorresponding to selection of the audio-only joining option (e.g., 642),the device joins the live communication session using audio transmittedfrom the electronic device (e.g., audio recorded by one or moremicrophones of the device), without transmitting video data from theelectronic device. In some embodiments, the device displays an indicatorthat the user has joined the live communication using audio only (e.g.,658 in FIG. 6L). This gives the user comfort in that video is not beingshared). Joining the live communication session using audio transmittedfrom the electronic device when prescribed conditions are met allows auser to join live communication sessions using only audio without theuser having to select or modify additional input and outputs of thedevice (e.g., turning off camera and/or turning on microphone).Performing an optimized operation when a set of conditions has been metwithout requiring further user input enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

At block 732, in response to receiving the second data indicating thatthe live communication session between the three or more participants isno longer active, the device ceases to display the notification (e.g.,614) or updates the notification to indicate that the live communicationsession is no longer active. In some embodiments, while the livecommunication session is active, the notification is updated to indicatethe current duration of the live communication session. In someembodiments, when the live communication session is no longer active,the notification ceases displaying the duration of the session anddisplays an indication that the session has ended (e.g., “call ended”).Ceasing to display the notification in response to receiving dataindicating that the live communication session is no longer activeprovides visual feedback to the user indicating that certain operationsassociated with the notification and/or the live communication sessionare no longer available. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In addition, ceasing to display the notification inresponse to receiving data indicating that the live communicationsession is no longer active automatically performs an operation when aset of conditions have been met without further user input. Performingan optimized operation when a set of conditions have been met withoutfurther user input enhances operability of the device and makes theuser-device interface more efficient (e.g., by decreasing unneededelements on the user interface and helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 736, 738, and 740, the device receives third dataindicating that a second live communication session is available betweentwo or more participants. In accordance with the second livecommunication session being available between two participants, thedevice displays a first type of notification (e.g., 672) (e.g., fullscreen notification) indicating that the second live communication isactive. In some embodiments, the first type of notification (e.g., 672)includes a menu (e.g., 630) with options for video (e.g., 634), audioonly (e.g., 643), and/or decline (e.g., 622). In accordance with thesecond live communication session being available between three or moreparticipants, the device displays a second type of notification (e.g.,676), different than the first type of notification, indicating that thesecond live communication is active. In some embodiments, the secondtype of a notification is a banner notification (e.g., 676) that is notfull screen or not persistently full-screen. Optionally, at block 742,the device initially displays the notification of the second type in afirst state (e.g., full screen state 672). Optionally, at step 744, thedevice transitions the notification (e.g., 672) of the second type to asecond state (e.g., 676) (e.g., a persistent, non-full screen state(e.g., a persistent banner positioned at the top edge of the display).Displaying a certain type of notification that a live communicationsession is active based on when prescribed conditions are met allows theuser to quickly recognize that the live communication is available whileproviding more and/or less decreasing distractions for the user based onthe prescribed condition (e.g., displaying a full screen notificationwhen there is a live communication session available between twoparticipants and displaying a banner when there is a live communicationsession available between two participants). Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 746, 748, 750, and 752, the device receives fourthdata indicating that a third live communication session is available. Inresponse to receiving the fourth data indicating that the third livecommunication session is available, in accordance with a determinationthat an active application (e.g., an application (instead of, forexample, a lock screen (e.g., 668) or home screen (e.g., 1400) of thedevice), such as a messaging application or a third party applicationprovided by an application developer other than the developer of theoperating system) is being displayed, the device displays a notification(e.g., 614 or 676) of a third type (e.g., a non-full screen state (e.g.,a banner positioned at the top edge of the display)) indicating that thethird live communication session is available. In response to receivingthe fourth data indicating that the third live communication session isavailable, in accordance with a determination that an active applicationis not being displayed (e.g., FIG. 6W), the device provides anotification of a fourth type (e.g., 678 or a full screen notification(e.g., 616)) indicating that the third live communication session isavailable. Displaying a certain type of notification that a livecommunication session based on when prescribed conditions are met allowsthe user to quickly recognize that the live communication is availablewhile providing more and/or less decreasing dissections for the userbased on the prescribed condition (e.g., displaying a full screennotification when the device is displaying an application and displayinga banner when the device is not displaying an application). Performingan optimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 754, 756, 758, and 760, the device receives fifthdata indicating that a fourth live communication session is available.In response to receiving fifth data indicating that the fourth livecommunication session is available, in accordance with a determinationthat a current context (e.g., a current operational mode of the device(e.g., whether the device is in a do-not-disturb (DND) mode; whethercertain applications or hardware are operationally active; whether thedevice is locked or unlocked; whether one or more sensors of the devicedetect that a user is looking at the display) of the electronic devicemeets alert issuance criteria, the device displays a notification (e.g.,676) that the fourth live communication session is available whileproviding a non-visual notification (e.g., 678) that the fourth livecommunication session is available (e.g., an audio or haptic output). Insome embodiments, the output or alert corresponds to the notification(e.g., 614). In response to receiving fifth data indicating that thefourth live communication session is available, in accordance with adetermination that the current context of the electronic device does notmeet the alert issuance criteria, the device displays the notification(e.g., 614) that the fourth live communication session is availablewithout providing the non-visual notification (e.g., 678) that thefourth live communication session is available. In some embodiments,while an alert of the first type is not issued, an alert of a secondtype is issued (e.g., a visual alert, such as a banner notification(e.g., 676)). In some embodiments, a relevant context of the device iswhether the device is in a locked or unlocked state. In someembodiments, when the device is locked, an alert of a first type (e.g.,an audio or haptic output) is issued; when the device is unlocked (e.g.,when a user is actively operating/interacting with the device), an alertis not issued (e.g., suppressed). In some embodiments, a relevantcontext of the device is whether the device detects (e.g., via one ormore sensors (e.g., cameras, infrared sensors) that a user is looking atthe display or not. In some embodiments, when the device is locked, analert of a first type (e.g., an audio or haptic output) is issued. Insome embodiments, when the device detects that a user is not looking atthe display (e.g., FIG. 6AA), an alert of the first type (e.g., 682) isgenerated. In some embodiments, when a user is activelyoperating/interacting with the device (e.g., FIG. 6AB), an alert of thefirst type is not generated (e.g., suppressed).)

In some embodiments, the alert issuance criteria include a requirementthat the device is locked in order for the alert issuance criteria to bemet. In some embodiments, the alert issuance criteria includes arequirement that the device does not detect the attention of a user onthe device in order for the alert issuance criteria to be met. (e.g.,based on detected user inputs with input devices of the electronicdevice, based on a detected gaze (e.g., 680) of the user via gazetracking, face tracking, or the like). Displaying a notification (e.g.,614) and determining whether to provide a non-visual notification basedon when prescribed conditions are met allows the user to quicklyrecognize that the live communication is available while providing moreand/or less interruption to the user based on the prescribed. Performingan optimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Note that details of the processes described above with respect tomethod 700 (e.g., FIGS. 7A-7F) are also applicable in an analogousmanner to the methods described below. For example, method 700optionally includes one or more of the characteristics of the variousmethods described below with reference to methods 900, 1100, 1300, and1500. For example, method 700 may launch a live communication sessionusing methods 1300 and 1500, and starting and running the livecommunication session may include methods 900 and 1100. For brevity,these details are not repeated below.

FIGS. 8A-8BK illustrate exemplary user interfaces for a livecommunication session, in accordance with some embodiments. The userinterfaces in these figures are used to illustrate the processesdescribed below, including the processes in FIG. 9A-9K.

FIG. 8A illustrates device 600 displaying messaging user interface 604for a message conversation between the group The Dream Team. A livecommunication session between the participants of the group is notactive (e.g., as indicated by the absence of live communicationaffordance 610). As illustrated in FIG. 8A, the header of messaging userinterface 604 is expanded to show video join affordance 626B (asdescribed above). As illustrated in FIG. 8A, device 600 receives userinput 850A (e.g., a tap on video join affordance 626B) corresponding toa request to display a user interface for a live communication sessionbetween the participants of the participant group The Dream Team. Asillustrated in FIG. 8A, the request includes a request to start a newlive video communication session. In some embodiments, the requestincludes a request to join an existing live communication session (e.g.,as described above with respect to FIG. 6F).

In response to user input 650A, device 600 displays message 800 inmessage region 606 and indication 802 that the live communicationsession is being initiated. In some embodiments, message 800 includessome of the characteristics of notification 614 described above. Oncethe live communication session is initiated, device 600 displays livecommunication user interface 804, which concurrently includes aplurality of representations of participants in the live communicationsession.

In accordance with a determination that the number of participants inthe live communication session includes more than a threshold number ofparticipants (e.g., more than five participants, including the user ofdevice 600), device 600 concurrently displays representations of a firstplurality of the participants other than a user of device 600 at one ormore sizes (e.g., three different sizes) greater than a respectivethreshold size while displaying representations of a second plurality ofthe participants in an overflow region, where the second plurality ofparticipants includes one or more participants not included in the firstplurality of participants and the representations of the one or moreother participants displayed in the overflow region are displayed at asize smaller than the respective threshold size.

In the embodiments illustrated in FIGS. 8A-8BK, the threshold number ofparticipants is five participants (including the user of device 600).Since the group The Dream Team includes fifteen participants, device 600determines that the number of participants in the live communicationsession includes more than the threshold number of participants. Asillustrated in FIG. 8C, in accordance with this determination, livecommunication user interface 804 includes a 4-on-1 layout of fourprimary representations 810A-810D (collectively 810) of four respectiveparticipants other than the user of device 600 displayed in canvasregion 811, user representation 806 of a view of a camera of device 600(corresponding to the user of device 600), and overflow region 820 withoverflow representations 822A-822J (collectively 822) corresponding tothe participants of the live communication session other than the userof device 600 and the participants represented in canvas region 811, asubset of which (e.g., 822A-822D) is displayed at any one time. In someembodiments, the layout is one of a plurality of predefined layouts inwhich representations of participants are anchored at differentlocations in user interface 804. FIG. 8AW illustrates another embodimentof user interface 804. In the embodiment illustrated in FIG. 8AW,overflow region 820 includes overflow representations for all of theparticipants of the live communication session other than the user ofdevice 600, such that participants represented in canvas region 811 arealso represented in overflow region 820.

If the number of participants in the group is less than or equal tofive, then in accordance with a determination that the number ofparticipants in the live communication session is less than or equal tofive, device 600 displays primary representations of the participantsand a representation of the user of device 600 without displayingoverflow representations in an overflow region.

Each of primary representations 810 is displayed at one of threepredetermined representation sizes (small, medium, and large). Primaryrepresentation 810A is displayed at the small size, primaryrepresentation 810B is displayed at the large size, and primaryrepresentation 810C and primary representation 810D are displayed at themedium size. In FIG. 8C, each of primary representations 810 overlapswith at least one other primary representation. Overflow representations822 are all displayed at the same size, which is smaller than the smallrepresentation size associated with primary representations 810. In theillustrated embodiment, the respective threshold size referred to aboveis greater than the size of the overflow representations 822 and smallerthan the small representation size for the primary representations 810.User representation 806 is the same size as the overflow representations822 and is aligned with or displayed within overflow region 820.

As illustrated in FIG. 8C, live communication session user interface 804displays three full overflow representations and one partial overflowrepresentation at one time. The remaining overflow representations arehidden, and can be displayed by scrolling overflow region 820 (e.g.,either in response to user input or a detected event such as aparticipant actively participating by talking or moving). In FIG. 8C,overflow representation 820D is partially shaded to indicate that moreoverflow representations exist and/or that overflow region 820 can bescrolled. In some embodiments, the order of overflow representations 822in overflow region 820 depends on the order in which participants jointhe live communication session. In some embodiments, the order ofoverflow representations 822 in overflow region 820 changes over time(e.g., as participants join or leave the live communication session oras participants move in or out of canvas region 811 due to, e.g., userselection of overflow representations or active participation byparticipants represented in the overflow region, as described in greaterdetail below). In some embodiments, the order of the overflowrepresentations in overflow region 820 is fixed (e.g., the order inwhich overflow representations are displayed does not change over time).In some embodiments, the order of the overflow representations is basedon the order of a list associated with a group (e.g., the order in whichthe participants were added to the group when the group was created). Insome embodiments, the display (e.g., live video feed) of primaryrepresentations 810 is updated at a higher rate than display of overflowrepresentations 822.

FIG. 8C illustrates live communication session interface 804 shortlyafter initiation of the live communication session and before any of theparticipants (other than the participant associated with device 600) hasjoined the live communication session. As illustrated in FIG. 8C,primary representations 810 and overflow representations 822 includeplaceholders for participants that are not connected with the livecommunication session. As illustrated in FIG. 8C, the placeholdersinclude a representation with an indication of the participant's name(e.g., first name or first and last initial) and a colored object. Insome embodiments, the color and/or pattern of each colored object isdifferent or is selected from a set of predefined colors and/orpatterns. Optionally, the placeholders include a status indicatorindicating that the participant is waiting to connect to the livecommunication session, and a reminder affordance (“RING”) for sending areminder to the corresponding participant that the live communicationsession is available. In some embodiments, reminder affordances are notinitially included on the placeholders and are displayed in accordancewith a determination that the corresponding participant has not joinedthe live communication session after a predetermined period of time.

As illustrated in FIG. 8C, device 600 receives (e.g., detect) user input850A (e.g., a tap) corresponding to selection of primary representation810C. As illustrated in FIG. 8D, in response to receiving user input850A, device 600 modifies primary representation 810C to includeadditional identifying information (e.g., the participant's first name,Marcel) and reminder affordance 808A for sending a reminder to thecorresponding participant that the live communication session isavailable.

FIG. 8D also shows that the participant corresponding to primaryrepresentation 810B joins the live communication session. In someembodiments, device 600 receives data indicating that the participantcorresponding to primary representation 822B has joined the livecommunication session. As illustrated in FIG. 8D, in response to theparticipant corresponding to primary representation 822B joining thelive communication session, device 600 replaces the placeholder ofprimary representation 822B with live video data from the correspondingparticipant. In some embodiments (e.g., FIG. 8AX), device 600 replacesoverflow representation 822B corresponding to the participant with livevideo data from the participant. As used herein, the phrase “live videodata from a participant” (or equivalents thereof) includes live videodata transmitted by a device (e.g., smartphone, laptop computer, desktopcomputer, or tablet computer) associated with the participant or by adevice associated with an account associated with the participant (e.g.,an email account or cloud storage account).

As illustrated in FIG. 8D, device 600 receives (e.g., detects) userinput 850B (e.g., a tap) corresponding to selection of reminderaffordance 808. In response to input 850B, device 600 sends instructionsto cause a new notification (e.g., call, ring, etc.) indicating that thelive communication session is available at a device associated with thecorresponding participant, where the new notification is different thana previous notification sent to the participant. In some embodiments,the new notification is more intrusive than the previous notification(e.g., an audio ring, haptic output, or full screen notification, ascompared to a banner notification without audio or haptic output). InFIG. 8D, device 600 also displays reminder affordance 808B on primaryrepresentation 810D to indicate that the participant corresponding toprimary representation 810D is currently being reminded that the livecommunication session is available (e.g., due to a reminder sent fromanother participant currently connected to the live communicationsession, such as the participant represented by primary representation810B or a reminder sent previously by the user of device 600). Device600 also modifies reminder affordance 808A to indicate that thecorresponding participant is being notified (e.g., by replacing “RING”with “RINGING”, as illustrated in FIG. 8E).

As illustrated in FIG. 8E, in accordance with a determination that apredetermined period of time has elapsed since the live communicationsession started, device 600 ceases to display overflow representations822 and increases the size of user representation 806. In someembodiments, primary representation 810 are resized and/or moved whenoverflow representations 822 are removed. In the illustrated embodiment,primary representation 810D is moved down. In some embodiments, device600 ceases to display overflow representations 822 in accordance with adetermination that the number of participants currently joined in thelive communication session is less than or equal to the threshold numberof participants (e.g., five participants, including the user of device600).

At the time corresponding to FIG. 8E, there are two participants joinedin the live communication session. In some embodiments, after a furtheramount of time, device 600 ceases to display one of the primaryrepresentations 810 and changes the layout of live communication userinterface 804 to a 3-on-1 layout. In some embodiments, the 4-on-1 layoutand the 3-on-1 layout are in a set of predetermined layouts that havesimilar relative placement of primary representations 810. In someembodiments, having a relatively similar placement of primaryrepresentations includes a placement in which a primary representationremains in a similar position on user display 602 when there is atransition between layouts within the set (e.g., the position of arepresentation in one layout overlaps the position of the representationin another layout of the same set).

In some embodiments, device 600 determines which primary representationto remove based on whether the corresponding participant has joined thelive communication session (e.g., a representation is not removed if thecorresponding participant has joined), whether a participant has beenreminded (e.g., a representation of a participant that has been remindedwill not be removed until a predetermined amount of time after selectionof the corresponding reminder affordance), the placement of aparticipant in the list of participants (e.g., a representation of aparticipant earlier in the list is removed before a representation of aparticipant later in the list), and the placement of the representationon live communication user interface 804 (e.g., representations closerto the top of display 602 are removed before representations lower ondisplay 602). As illustrated in FIG. 8F, device 600 ceases to displayprimary representation 810A and moves primary representation 810B up(e.g., to reduce the overlap between representations 810B and 810C),while maintaining and the relative vertical order of the remainingrepresentations (e.g., representation 810B is larger than and aboverepresentation 810C, and representations 810C is above and the same sizeas representation 810D). FIG. 8F also illustrates that reminderaffordance 808A is modified to “RING AGAIN”, which indicates that thecorresponding participant has been reminded at least one time and is notcurrently being reminded. Reminder affordance 808B is also modified(e.g., from “RINGING” to “RING” and a different color) to indicate thatthe corresponding participant has previously been reminded, but not bythe user of device 600.

At the time corresponding to FIG. 8F, there are still only twoparticipants joined in the live communication session. In someembodiments, after a further amount of time, device 600 ceases todisplay another primary representation 810 and changes the layout oflive communication user interface 804 to a 2-on-1 layout (e.g., in thesame set of predetermined layouts at the 4-on-1 and 3-on1 layoutsillustrated in FIGS. 8E-8F). As illustrated in FIG. 8G, device 600ceases to display primary representation 810D and enlarges primaryrepresentations 810B and 810C. FIG. 8G illustrates a three participant(e.g., 2-on-1) layout in which the primary representations of the twoparticipants besides the user of device 600 are equal size andvertically aligned one above the other. The vertical order of primaryrepresentations 810B and 810C is maintained, with primary representation810B above primary representation 810C. In the layout of FIG. 8F, thereis no overlap between primary representations, and user representation806 overlaps primary representation 810C.

At the time corresponding to FIG. 8G, there are still only twoparticipants joined in the live communication session. In someembodiments, after a further amount of time, device 600 ceases todisplay another primary representation and changes the layout of livecommunication user interface 804 to a 1-on-1 layout. As illustrated inFIG. 8H, device 600 ceases to display primary representation 810C andenlarges primary representation 810B to a full screen size. In someembodiments, device 600 changes from the layout of FIG. 8G to the layoutof FIG. 8H a predetermined amount of time after selection of reminderaffordance 808A. FIG. 8H illustrates a two-participant (e.g., 1-on-1)layout in which the remaining primary representation 810B is displayedin full screen and is overlapped by user representation 806. Asillustrated in FIG. 8H, remaining primary representation 810B and userrepresentation 806 are vertically elongated rectangles (e.g., in profileorientation), compared to the square shapes of the primaryrepresentations 810 and user representation 806 in the 4-on-1, 3-on-1,and 2-on-1 layouts illustrated in FIGS. 8E-8G, respectively.

Turning to FIG. 8I, a third participant, Stephen, joins the livecommunication session. As illustrated in FIG. 8I, in response toparticipant Stephen joining the live communication session, device 600transitions from display of the 1-on-1 layout of FIG. 8H to the 2-on-1layout, with primary representation 810E of the most recently joinedparticipant above representation 810B of the existing participant (otherthan the user of device 600). In some embodiments, device 600 displaysnotification 812 and/or generates an audio or haptic output indicatingthat a participant has joined the live communication session, asillustrated in FIG. 8I. In some embodiments, since a new primaryrepresentation is displayed when participant Stephen joins the livecommunication session, device 600 refrains from displaying notification812 or generating an audio or haptic output, as primary representation810E serves as an indication that a participant has joined the livecommunication session.

Participant Stephen joins the live communication session in an audioonly mode and provides live audio data without video. The audio-onlycommunication status of participant Stephen is indicated by primaryrepresentation 810E and notification 812. Primary representation 810Eindicates the audio-only communication status by displaying theparticipant's name and an avatar associated with the participant.Notification 812 indicates the communication status with text (“JOINEDTHIS CALL WITH AUDIO”). As illustrated in FIG. 8J, device 600 ceases todisplay notification 812 (e.g., after a predetermined amount of timeafter participant Stephen joins the live communication session).

In some embodiments, in response to receiving data indicating that aparticipant that is joined via audio only is actively participating(e.g., speaking), device 600 displays a visual indication on therepresentation of the participant who is speaking. In FIG. 8J, therepresentative image (e.g., avatar) within primary representation 810Eis animated (e.g., with “voice circles”) to indicate that participantStephen is speaking.

Turning to FIG. 8K, a fourth participant, Marcel, joins the livecommunication session. As illustrated in FIG. 8K, in response to thefourth participant joining the live communication session, device 600transitions from the 2-on-1 layout to the 3-on-1 layout (e.g., asillustrated in FIG. 8F), with primary representation 810F of the mostrecently joined participant above primary representations 810B and 810Eof the existing participants (other than the user of device 600).

Participant Marcel joins the live communication session in a video modeand provides live video and audio data. The video communication statusof participant Marcel is indicated by primary representation 810F.Primary representation 810F indicates the video communication status bydisplaying a live video stream. As illustrated in FIG. 8K, device 600does not immediately display a notification indicating that participantMarcel has joined the live communication session.

Turning to FIG. 8L, a fifth participant, Hoan, joins the livecommunication session. As illustrated in FIG. 8L, in response toparticipant Hoan joining the live communication session, device 600transitions from the 3-on-1 layout to the 4-on-1 layout, with primaryrepresentation 810G of the most recently joined participant as thesmallest primary representation and positioned above and to the right ofprimary representation 810F. In some embodiments, in accordance with adetermination that participants Marcel and Hoan both joined the livecommunication session within a predetermined window of time, device 600displays combined notification 816 indicating that participants Marceland Hoan have joined the live communication session, as illustrated inFIG. 8L. In some embodiments, combined notification 816 is displayed inaccordance with a determination that the participants joined with thesame communication status (e.g., video).

As indicated by the live video stream displayed in primaryrepresentation 810G and the representation 816A of the video camera innotification 816, participant Hoan joins the live communication sessionin a video mode such that device 600 receives live video and audio dataassociated with participant Hoan.

As illustrated in FIG. 8L, device 600 receives (e.g., detects) userinput 850C (e.g., a double tap) on display 602 at a location of primaryrepresentation 810F. In some embodiments, as illustrated in FIG. 8M, inresponse to receiving user input 850C, device 600 enlarges primaryrepresentation 810F to the largest representation size and reducesprimary representation 810E to a medium size (while maintaining the4-to-1 layout). As illustrated in FIG. 8M, device 600 ceases to displaynotification 816 (e.g., after a predetermined amount of time afterparticipant Hoan joins the live communication session).

In some embodiments, the configuration of a layout is dynamicallychanged in response to receiving data indicating that a participant isactively participating in the live communication session. In FIG. 8M,device 600 receives data indicating that participant Hoan, correspondingto primary representation 810G is actively participating. As illustratedin FIG. 8M, in response to receiving data indicating that theparticipant corresponding to primary representation 810G is activelyparticipating, device 600 enlarges primary representation 810G from thesmall representation size to a medium representation size and reducesprimary representation 810B from a medium representation size to thesmall representation size. In this way, the 4-on-1 layout maintains onesmall sized representation, two medium sized representations, and onelarge sized representation.

As illustrated in FIG. 8M, device 600 receives (e.g., detects) userinput 850D (e.g., a double tap) on display 602 at a location of primaryrepresentation 810G. In some embodiments, as illustrated in FIG. 8N, inresponse to receiving user input 850D, device 600 enlarges primaryrepresentation 810G and brings primary representation 810G to theforeground in front of the other displayed primary representations. Thebackground behind enlarged primary representation 810G is greyed out,except for user representation 806.

As illustrated in FIG. 8O, device 600 receives (e.g., detects) userinput 850E (e.g., a tap) on cancel affordance 818. As illustrated inFIG. 8P, in response to receiving user input 850E, device 600 reducesthe size of primary representation 810G and returns the display to the4-on-1 layout displayed prior to user input 850D, except with primaryrepresentation 810G occupying the large size primary representationinstead of primary representation 801F, which is reduced to a mediumrepresentation size.

Turning to FIG. 8Q, a sixth participant joins the live communicationsession. As illustrated in FIG. 8Q, in response to the sixth participantjoining the live communication session, and in accordance with thenumber of connected participants exceeding the predetermined thresholdof five participants, device 600 displays overflow representation 822Aof the sixth participant in overflow region 820 and moves userrepresentation 806 to overflow region 820, with overflow representation822A to the left of user representation 806. As illustrated in FIG. 8Q,overflow representation 822A and user representation 806 are centrallyaligned in overflow region 820. In some embodiments, device 600 replacesthe smallest of the primary representations (e.g., 810B) with arepresentation of a newly joined participant (e.g., the sixthparticipant) and displays an overflow representation of the replacedparticipant in overflow region 820 (e.g., representation 810B is reducedin size and moved to overflow region 820 in the location of overflowrepresentation 822A). In some embodiments, in response to the sixthparticipant joining the live communication session, and in accordancewith the number of connected participants exceeding the predeterminedthreshold of five participants, device 600 displays overflowrepresentations of the currently joined participants or all of theparticipants (currently joined and invited) in the overflow region, asillustrated, e.g., in FIG. 8AY.

As illustrated in FIG. 8Q, the overflow region 820 and the primaryrepresentations 810 do not overlap. In some embodiments, to make roomfor the overflow region, device 600 reduces the size and/or verticallytranslates one or more of the primary representations. In someembodiments, device 600 reduces the size of the medium sized primaryrepresentations and maintains the size of the small and large primaryrepresentations. In some embodiments, device 600 reduces the verticalseparation and/or increases the vertical overlap between the primaryrepresentations.

As illustrated in FIG. 8R, a seventh participant joins the livecommunication session. In response to the seventh participant joiningthe live communication session, device 600 adds overflow representation822B of the seventh participant in overflow region 820 to the left ofoverflow representation 822A. As illustrated in FIG. 8R, overflowrepresentation 822A and user representation 806 are moved to the right(compared to FIG. 8Q) such that overflow representations 822A and 822Band user representation 806 are centrally aligned in overflow region820. In some embodiments, overflow representation 822B of the seventhparticipant is added to the right of overflow representation 822A,between overflow representation 822A and user representation 806. In theembodiment illustrated in FIG. 8AZ, in response to the seventhparticipant joining the live communication session, device 600 replacesthe smallest primary representation (e.g., 810H in FIG. 8AY) withprimary representation 810I, and changes overflow representation 822Fcorresponding to participant Pablo from a placeholder to a live videostream.

As illustrated in FIG. 8S, an eighth participant joins the livecommunication session. In response to the eighth participant joining thelive communication session, device 600 adds overflow representation 822Cof the eighth participant in overflow region 820 to the left of overflowrepresentation 822B. As illustrated in FIG. 8S, overflow representations822A and 822B and user representation 806 are moved to the right(compared to FIG. 8Q) such that overflow representations 822A, 822B, and822C and user representation 806 are centrally aligned in overflowregion 820.

As illustrated in FIG. 8T, a ninth participant joins the livecommunication session. In response to the ninth participant joining thelive communication session, device 600 adds overflow representation 822Dof the ninth participant in overflow region 820 to the left of overflowrepresentation 822C. As illustrated in FIG. 8T, overflow representations822D, 822C, and 822B and user representation 806 are fully displayed.Overflow representation 822A is partially displayed, with the right sideportion overlapped by user representation 806. Overflow representation822A is shaded (e.g., with a shadow effect) where overflowrepresentation 822A meets user representation 806. In the illustratedembodiment, the size of overflow representations 822 and userrepresentation 806 remain the same as the seventh, eighth, and ninthparticipants join the live communication session. FIG. 8T illustratesthat device 600 displays combined notification 824 indicating thatparticipants Mary, Mick, John, and Tom have joined the livecommunication (represented by overflow representations 822A, 822B, 822C,and 822D, respectively). In some embodiments, as additional participants(e.g., beyond the ninth participant) join the live communicationsession, overflow representations of newly joined participants are addedto the left end of the existing overflow representations, with theexisting overflow representations shifting to the right in overflowregion 820 to make room for the new overflow representation and userrepresentation 806 remaining fully displayed at the right side ofoverflow region 820 (e.g., existing overflow representations arescrolled to the right “underneath” user representation 806). In someembodiments, overflow representations of newly joined participants areadded to the right end of the existing overflow representations, withuser representation 806 remaining fully displayed at the right side ofoverflow region 820. In some embodiments, if the overflow region isfully occupied, a representation of a newly joined participant added tothe right end of the existing overflow representations is notimmediately displayed (e.g., the new overflow representation isdisplayed in response to scrolling the overflow representations to theleft).

Turning to FIG. 8U, device 600 receives data indicating that theparticipant corresponding to overflow representation 822B activelyparticipates in the live communication session. In some embodiments,device 600 replaces a primary representation with a representation ofthe actively participating participant in response to a determinationthat the activity level of the actively participating participant hasincreased above the activity level of a participant represented by aprimary representation. In the illustrated embodiment, in response toreceiving data indicating that the participant corresponding to overflowrepresentation 822B actively participates in the live communicationsession, device 600 replaces the smallest primary representation with arepresentation of the actively participating participant and displays anoverflow representation of the participant corresponding to the replacedprimary representation. As illustrated in FIG. 8U, primaryrepresentation 810B (the smallest primary representation) is shaded orbegins to fade out; overflow representation 822B of the activelyparticipating participant is shaded or begins to fade and is reduced insize; and overflow representations 822C and 822D are shifted to theright as new overflow representation 822E corresponding to theparticipant of primary representation 810B (the primary representationbeing replaced) enters overflow region 820 from the left edge of display602 to the left of overflow representation 822D. In some embodiments,primary representation 810B is slightly enlarged (e.g., to emphasizethat it is being replaced). As illustrated in FIGS. 8V-8W, primaryrepresentation 810B is replaced with primary representation 810H of theactively participating participant; overflow representation 822Bcontinues to shrink until it is removed; and overflow representations822C and 822D continue shifting to the right until overflowrepresentation 822E is fully displayed and overflow representation 822Coccupies the position of overflow region 820 previously occupied byremoved overflow representation 822B. In FIGS. 8U-8W, overflowrepresentation 822A and user representation 806 remain in the samepositions.

In some embodiments, in response to the user of device 600 activelyparticipating, device 600 visually distinguishes user representation806. In the embodiment illustrated in FIG. AZ, device 600 displays abold border around user representation 806 and highlights the initialsbar at the bottom of user representation 806.

Turning to FIG. 8X, device 600 is rotated from a portrait (vertical)orientation to a landscape (horizontal) orientation. As illustrated inFIG. 8X, in response to device 600 being rotated, device 600 maintainsthe representations in the respective areas of display 602 occupiedprior to rotation of device 600 (e.g., the user representation and eachprimary and overflow representation occupies the same area on display602 before and after rotation of device 600). The images within therepresentations are rotated so that the participants appear uprightrelative to the landscape orientation. In some embodiments, device 600displays an animated rotation of the primary representations or theimages within the primary representations (e.g., 90 degrees opposite thedirection of rotation of device 600). In some embodiments in which theoverflow representations are not squares, video data of a participant iscropped and/or translated to include (e.g., center) a user's face in therespective overflow representation when device 600 is rotated. In someembodiments in which the primary representations are not squares, videodata of a participant is cropped and/or translated to center a user'sface in the respective primary representation when device 600 isrotated. In some embodiments, video data of a participant is cropped toinclude a user's face (e.g., regardless of whether or not device 600 isrotated, such as when received video data has a different aspect ratioor format than the representation in which it is displayed).

As illustrated in FIG. 8X, the participant corresponding to primaryrepresentation 810H begins actively participating. In response, device600 enlarges primary representation 810H and reduces the size of primaryrepresentation 810E, as illustrated in FIG. 8Y. In the embodimentillustrated in FIGS. 8BA-8BB, device 600 visually distinguishes overflowrepresentation 822F (e.g., emphasized initials bar) corresponding to theactive participant.

In some embodiments, in response to a participant joining the livecommunication session, device 600 replaces an existing primaryrepresentation with a primary representation of the newly joinedparticipant and moves the replaced participant to overflow region 820.In FIG. 8Z, a participant Pablo joins the live communication session. Inresponse, device 600 replaces the smallest primary representation 810E(Stephen) with primary representation 810I of newly joined participantPablo and displays overflow representation 822F (Stephen) in overflowregion 820. To make room for overflow representation 822F, overflowrepresentations 822A, 822C, 822D, and 822E are scrolled (e.g., upwardaccording to the orientation of device 600 illustrated in FIG. 8Z). Asillustrated in FIG. 8Z, overflow representation 822A is no longervisible and overflow representation 822C is partially hidden by userrepresentation 806.

As illustrated in FIG. 8AA, device 600 is rotated from a landscape(horizontal) orientation to a portrait (vertical) orientation. Inresponse, device 600 rotates the images of the participants in therepresentations while maintaining the area on display occupied by eachrepresentation (e.g., device 600 reverses the rotation performed inresponse to device being rotated from a portrait orientation to alandscape orientation).

In FIG. 8AB, the participant corresponding to primary representation810G leaves (disconnects from) the live communication session.Optionally, as illustrated in the embodiment shown in FIG. 8BC, inresponse, device 600 displays notification 828 indicating that theparticipant has left the live communication session. Since the number ofparticipants remains greater than five, device 600 maintains the 4-on-1layout with the display of overflow region 820. As illustrated in FIG.8AC, primary representation 810G is replaced with primary representation8107 of the participant with the next highest activity level, Stephen,and overflow representation 822F corresponding to participant Stephen isremoved from overflow region 820. When overflow representation 822F isremoved, the remaining overflow representations 822A, 822C, 822D, and822E are scrolled to the left. In the embodiment illustrated in FIG.8BD, overflow representation 822D of participant Hoan is updated todisplay a placeholder indicating that participant Hoan is not connectedand a reminder affordance to request participant Hoan to rejoin.

In some embodiments, if the number of participants in the livecommunication is reduced from six to five when a participant leaves thelive communication session, device 600 ceases to display overflow region820 and, optionally, increases the size and/or vertical spacing of oneor more of the primary representations.

In FIG. 8AC, device 600 receives (e.g., detects) user input 850F (e.g.,a double tap) corresponding to selection of overflow representation 822Eof participant Allison. As illustrated in FIG. 8AD, in response toreceiving user input 850F, device 600 displays enlarged representation830 of participant Allison in front of the 4-on-1 layout of thecurrently displayed primary representations 810. Display of enlargedrepresentation 830 is similar to the display of enlarged representation810G in FIG. 8N, except that enlarged representation 830 is not broughtforward from a primary representation in the 4-on-1 layout.

As illustrated in FIG. 8AD, device 600 receives (e.g., detects) userinput 850G (e.g., a tap) corresponding to selection of close affordance818. As illustrated in FIGS. 8AE-8AF, in response to receiving userinput 850F, device 600 displays primary representation 810K ofparticipant Allison in place of primary representation 8107 ofparticipant Stephen (e.g., the participant having the lowest activitystatus of the participants with a current primary representation).Primary representation 810K (at least initially) includes ribbon (orsash) 842A along the bottom of edge, which includes the participant'sname and an affordance for enlarging the representation (e.g., asillustrated in FIG. 8AD). FIGS. 8AE-8AF illustrate an exemplaryanimation of replacing primary representation 8107 with primaryrepresentation 810K. Primary representation 810K is initially displayedat the small representation size (like primary representation 8107 inFIG. 8AC) and then is enlarged to a medium representation size, whileprimary representation 810H is reduced from medium to small. Overflowrepresentation 822E corresponding to participant Allison is reduced insize and/or fades out, while overflow representation 822F of thereplaced participant translates into overflow region 820. In someembodiments, ceasing to display primary representation 8107 anddisplaying primary representation 810K includes a crossfade from primaryrepresentation 8107 to primary representation 810K and/or a tint tocreate a “flash” effect. FIGS. 8BE-8BH illustrate an embodiment in whichprimary representation 8107 is removed with a flash effect.

In some embodiments, overflow region 820 is scrollable. Referring to theembodiment illustrated in FIGS. 8BH-BI, device 600 scrolls the overflowrepresentations 822 in overflow region 820 in response to activeparticipation by a participant for which the corresponding overflowrepresentation is not currently displayed or not completely displayed.As illustrated in FIG. 8BH, overflow representation 822F of theparticipant (corresponding to primary representation 810I) is partiallyhidden in overflow region 820. In FIG. 8BH, the participantcorresponding to primary representation 810I actively participates inthe live communication session. As illustrated in FIG. 8BI, in responseto receiving data indicating that the participant corresponding toprimary representation 810I is actively participating, device 600enlarges primary representation 810I, scrolls the overflowrepresentations 822 such that overflow representation 822F is fullydisplayed (e.g., near the middle of the displayed portion of overflowregion 820), and visually indicates overflow representation 822F byhighlighting the initial bar at the bottom of overflow representation822F.

Turning to FIG. 8AG, device 600 detects contact 850H, which is movedhorizontally from right to left on overflow region 820 while maintainingcontact with display 602. As illustrated in FIG. 8AH, in response toreceiving contact 850H, device 600 scrolls overflow representations 822to the left on display 602 such that overflow representation 822F ismoved partially off the left edge of display 602 and overflowrepresentation 822A is no longer overlapped by user representation 806.In some embodiments, contact 850H and overflow representations 822 moveby the same horizontal amount.

Primary representations 810 remain unchanged while scrolling overflowregion 820. Also, user representation 806 remains in overflow region820.

In the embodiment illustrated in FIGS. 8BJ-8BK, contact 850H begins atthe left side of overflow region 820 and is moved horizontally to theright. In response, device 600 scrolls overflow representations 822 tothe right on display 602. In FIGS. 8BJ-8BK, device 600 removes (e.g., nolonger displays) representations from display 602 as they reach userrepresentation 806.

Turning to FIG. 8AH, device 600 receives (e.g., detects) user input 850I(e.g., a tap) at a location on display 602 where a primaryrepresentation 810 is not displayed. As illustrated in FIG. 8AI, inresponse to receiving user input 850I, device 600 displays call controlmenu 832, which includes call affordance 834 (e.g., to leave the livecommunication session), effects affordance 836, menu affordance 838, andhandle affordance 840.

As illustrated in FIG. 8AI, call control menu 832 does not overlap withoverflow region 820 and primary representations 810. To make room forcall control menu 832, device 600 reduces the size and/or verticallytranslates one or more of the primary representations 810 (e.g., reducesthe size of canvas region 811) and moves overflow region 820 upward ondisplay 602. In some embodiments, device 600 reduces the size of mediumsized primary representations and maintains the size of small and largeprimary representations. In some embodiments, device 600 reduces thevertical separation and/or increases the vertical overlap between theprimary representations.

In response to receiving user input 850I, device 600 also displaysribbons 842A-842D on each of the primary representations. Ribbons842A-842D include the name of the corresponding participant and anaffordance to enlarge the representation (e.g., as illustrated in FIG.8O). In some embodiments, the vertical height of ribbons 842A-842D isthe same as the amount of vertical overlap between primaryrepresentations.

As illustrated in FIG. 8AJ, device 600 receives (e.g., detects) userinput 850J (e.g., a tap) corresponding to selection of menu affordance838. In some embodiments, user input 850J includes an upward swipestarting at call control menu 832 (e.g., on handle affordance 840). Asillustrated in FIG. 8AK, in response to user input 850J, device 600expands call control menu 832 to display additional information andcontrols related to the live communication session, analogous to theadditional information and controls described with respect to callcontrol menu 630 in FIG. 6H.

As illustrated in FIG. 8AL, device 600 receives (e.g., detects) userinput 850K (e.g., a tap) corresponding to selection of effectsaffordance 836. As illustrated in FIG. 8AM, in response to receivinguser input 850K, device 600 displays an enlarged image 843 of the userof device 600 from camera 603 and modifies call control menu 832. Inmodified call control menu 832, effects affordance 836 is highlightedand effect options affordances 844A-844E are displayed. In FIG. 8AN,device 600 receives (e.g., detects) user input 850L (e.g., a tap)corresponding to selection of effect options affordance 844A. Asillustrated in FIG. 8AO, in response to receiving user input 850L,device 600 modifies call control menu 832 to include avatar optionsaffordances 846A-846C. In FIG. 8AO, device 600 receives (e.g., detects)user input 850M (e.g., a tap) corresponding to selection of avataraffordance 846C. As illustrated in FIG. 8AP, in response to receivinguser input 850M, device 600 displays effect 843A over the user's face inimage 843 and scrolls call control menu 832 to place affordance 846C ofthe currently selected avatar centered in call control menu 832. In FIG.8AP, device 600 receives (e.g., detects) user input 850N (e.g., a tap)on cancel affordance 848. As illustrated in FIG. 8AQ, in response toreceiving user input 850N, device 600 returns call control menu 832 tothe configuration of FIG. 8AN and maintains display of effect 843A onenlarged image 843. In FIG. 8AQ, device 600 receives (e.g., detects)user input 850O (e.g., a tap) on display 602 outside of enlarged image843 and call control menu 832. As illustrated in FIG. 8AR, in responseto receiving user input 850O, device 600 ceases displaying enlargedimage 843 and displays effect 843A on the image of the user in userrepresentation 806. When effect 843A is activated, as in FIG. 8AR, thelive video stream of the user of device 600 includes effect 843A.

As illustrated in FIG. 8AR, call control menu 832 includes mediaaffordance 844E. Device 600 receives (e.g., detects) user input 850P(e.g., a tap) corresponding to selection of media affordance 844E. Asillustrated in FIG. 8AS, in response to receiving user input 850P,device 600 modifies call control menu 832 to include media optionsaffordances 852A-852D. In FIG. 8AS, device 600 receives (e.g., detects)user input 850Q (e.g., a tap) corresponding to selection of mediaoptions affordance 852A (SKETCH). In response to receiving user input850Q, device 600 displays a representation of media content associatedwith media options affordance 852A in canvas region 811.

As illustrated in FIG. 8AT, in response to receiving user input 850Q,device 600 replaces primary representation 810K (e.g., the smallestprimary representation) with representation 810L of the selected mediaitem. In FIG. 8AT, device 600 receives (e.g., detects) user input 850R(e.g., a down swipe on call control menu 832) corresponding to a requestto dismiss call control menu 832. As illustrated in FIG. 8AU, inresponse to receiving user input 850R, device 600 ceases displaying callcontrol menu 832.

In some embodiments, selecting media options affordance 852A causes theassociated media content to be shared with other participants of thelive communication session (e.g., displayed in the canvas region on thedisplay of a device of another participant). In some embodiments,participants of the live communication session can interact with theshared media content (e.g., via a sequence of one or more inputs in theuser interface of the live communication application). FIG. 8AVillustrates an embodiment in which an additional feature 810L-1 has beenadded to the media content of representation 810L. In some embodiments,in response to participant interaction with the media content associatedwith representation 801L, device 600 enlarges representation 801L (e.g.,similar to the way in which a primary representation is enlarged thenselected or the participant corresponding to the representation activelyparticipates). As illustrated in FIG. 8AV, representation 810L isenlarged (compared to FIG. 8AT) in response to participant interactionwith associated media content. FIG. 8AV also illustrates representation810M of other content (e.g., a game between two participants) shared inthe live communication session.

FIGS. 9A-9K are a flow diagram illustrating a method in accordance withsome embodiments. Method 900 is performed at a device (e.g., 100, 300,500, or 600) with a display. Some operations in method 900 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 900 provides, among other things, anintuitive way for providing an adaptive and intelligent livecommunication user interface. In addition, among other things, method900 provides an intuitive way for users to interact with a livecommunication user interface. The method reduces the cognitive burden ona user by providing an adaptive and intelligent live communicationinterface based on the number of participants in the live communication,thereby creating a more efficient human-machine interface. Forbattery-operated computing devices, enabling a user to view a livecommunication user interface based on a predetermined number ofparticipants in the live communication is faster and more efficientlyconserves power and increases the time between battery charges.

At block 901, the device receives a request (e.g., 850A) to display auser interface for a live communication session between two or moreparticipants.

At block 902, in response to receiving the request to display the userinterface for the live communication session between two or moreparticipants, the device displays a live communication user interface(e.g., 804) including concurrently displaying a plurality ofrepresentations of participants (e.g., 810A-810D) in the livecommunication session, where displaying the plurality of representationsof participants in the live communication session includes blocks 902and 903. In some embodiments, the live communication session userinterface includes a placeholder (e.g., 810A of FIG. 8C) (e.g., agraphical placeholder (e.g., an image or virtual avatar) a textualplaceholder (e.g., a name or initials)) associated with a firstparticipant who has been invited to the live communication session buthas not yet joined the live communication session (e.g., a participantother than the user of the electronic device; a participant that has yetto join the communication session and that has been sent an invitationto join the communication session). In some embodiments, if aparticipant is sharing a live media stream that includes a live videostream, the representation includes images of the live video stream(e.g., 810B of FIG. 8D). In some embodiments, if a participant issharing a live media stream with only audio, the representation includesan avatar of the respective participant (e.g., 810E of FIG. 8I). In someembodiments, the plurality of representations of participants in thelive communication session includes a user communication status selectedfrom a group consisting of an audio-only status (e.g., participant iscommunication using audio only), a video status (e.g., participant istalking using video and audio), a video paused status (e.g.,participant's video is paused), a video not decoded status (e.g.,participants video stream cannot be processed because of latency issues,issues with the format of the video stream, etc.), a left status (e.g.,participant has left the live communication session), and a waiting tojoin (e.g., participant was invited to the live communication sessionbut have not joined the live communication yet) status.

At block 903, in accordance with a determination that the number ofparticipants in the live communication session includes more than athreshold number of participants, the device concurrently displaysrepresentations of a first plurality of the participants (e.g.,810A-810D) other than a user of the electronic device at one or moresizes greater than a respective threshold size while displayingrepresentations of a second plurality of the participants (e.g., 822) inan overflow region (e.g., 820), wherein the second plurality ofparticipants includes one or more participants not included in the firstplurality of participants and the representations of the one or moreother participants displayed in the overflow region are displayed at asize smaller than the respective threshold size. In some embodiments, acurrently active participant (e.g., 810E of FIG. 8J) (e.g., a mostactive participant, a participant that is speaking or moving in a mannerthat satisfies an active participant set of criteria) is displayed aspart of the first plurality of participants. In some embodiments, thecurrently active participant is displayed in a main region (e.g., 811)of the user interface (e.g., a region where the first plurality ofparticipants is displayed). In some embodiments, if a participant thatis not currently displayed in the main region (e.g., participantcorresponding to 822B of FIG. 8U) becomes the currently activeparticipant, the representation of that participant is displayed in themain region (e.g., 810H of FIG. 8V). In some embodiments, therepresentations of the first plurality of participants (e.g., 810) aredisplayed at two or more different sizes (e.g., with more activeparticipants or more recently active participants displayed at a largersize than less active participants or less recently active participants)and the representations of the second plurality of participants (e.g.,822) in the overflow region are displayed at the same size.

In some embodiments, the overflow region (e.g., 820) further includesrepresentations of the first plurality of the participants (e.g.,810A-810D) (e.g., representations of participants displayed in a mainregion of the user interface are also displayed in the overflow region)(e.g., FIG. 8AX). In some embodiments, the overflow region (e.g., 820)further includes placeholder representations (e.g., 822A in FIG. 8AW)(e.g., an avatar, a name, initials) for invitees of the livecommunication session that are not currently connected to the livecommunication session. Displaying representations of participantsdifferently based on whether prescribed condition as met (e.g., based onwhether the live communication session includes greater than a number ofparticipants) allows the user to view representations of participants inthe live communication session easier or more efficiently. Performing anoptimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 904 and 905, the device updates the display of therepresentations (e.g., the arrangement, positions, size, position,visual indications, visual content, and/or video feed of arepresentation) of the first plurality of the participants (e.g.,810A-810D) at a first rate (e.g., the representations of the firstplurality of participants are live video feeds that are updated at aframe rate that is at or greater than a typical video refresh rate suchas 24, 30, or 60 frames per second) and updates the display of therepresentations (e.g., the arrangement, positions, size, position,visual indications of the representations, visual content, and/or videofeed of a representation) of the second plurality of the participants(e.g., 822) in the overflow region at a second rate, where the secondrate is lower than the first rate.) (e.g., the thumbnails in theoverflow region are updated over time, but at a rate that is less thanthe video refresh rate of the representations of the plurality ofparticipants in canvas region 811, such as 1, 5, or 10 frames persecond). Updating the displays of sets of representations at differentrates allows the device to provide the user with feedback about thecurrent state of representations in a live communication whileconserving processing power to provide a set of representations at afaster rate than the other set of representations, and thereby, provingthe user the ability to view a higher prioritized set of representationsat a greater rate. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at block 906, the device, while displaying therepresentations of the second plurality of participants (e.g., 822) inthe overflow region (e.g., 820), displays a representation of a view ofa camera of the electronic device (e.g., 806) in the overflow region(e.g., a self-view camera view). Displaying a representation of a viewof a camera of the electronic device provides the user with visualfeedback about the current state of the view of the user camera (e.g.,what a user is showing to other participants in the live communicationsession). Providing improved visual feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 907, 908, 909, and 910, the device detects aninput (e.g., 850H) (e.g., a horizontal finger drag) corresponding to theoverflow region. In response to detecting the input corresponding to theoverflow region, the device scrolls the overflow region (e.g., scrollingthe representations of the second plurality of participants in a firstdirection in accordance with the input), to cease to display (e.g., tohide) a subset of the representations of the second plurality ofparticipants (e.g., 822G and 822H in FIGS. 8BJ-8BK). While scrolling theoverflow region, the device maintains the display of the representationof the view of the camera of the electronic device (e.g., 806) in theoverflow region (e.g., FIGS. 8BJ-BK). In some embodiments, scrolling theoverflow region includes displaying an animation of one or morerepresentations of the second plurality of participants appearing toscroll under the representation of the view of the camera of the device.Maintaining the display of the representation of the view of the cameraof the electronic device while scrolling the overflow region providesthe user with continued feedback about the current state of the view ofthe camera of the electronic device (e.g., what a user is showing toother participants in the live communication session) while alsoallowing the user with the ability to scroll through representation ofparticipants in the overflow region. Providing improved visual feedbackto the user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 911, 912, and 913, detects an input (e.g., 850F)(e.g., tap gesture corresponding to a representation in overflow region820 or a gesture having a characteristic intensity that exceeds anintensity threshold corresponding to a representation in overflow region820) corresponding to selection of a representation (e.g., 822E of FIG.8AC) in the representations of the second plurality of participants inthe overflow region. In response to detecting the input corresponding toselection of a representation in the representations of the secondplurality of participants in the overflow region, the devices replacesthe display of a representation (e.g., 810J in FIG. 8AC) in therepresentations of the first plurality of the participants with arepresentation of the participant (e.g., 810K of FIG. 8AE) correspondingto the selected representation in the overflow region (e.g., replace thelast recently active participant in the main display). In someembodiments, replacing the display of the representation in therepresentations of the first plurality of the participants with therepresentation of the participant corresponding to the selectedrepresentation in the overflow region includes ceasing to display theselected representation in the representations of the second pluralityof participants in the overflow region. In some embodiments, ceasing todisplay the selected representation in the overflow region includes ananimation of the selected representation shrinking. In some embodiments,other representations in the overflow region are translated to fill agap where the selected representation was displayed. In someembodiments, replacing the display of the representation in therepresentations of the first plurality of the participants with therepresentation of the participant corresponding to the selectedrepresentation in the overflow region includes displaying therepresentation of the participant corresponding to the selectedrepresentation at a first size and then enlarging the representation ofthe participant corresponding to the selected representation to a secondsize larger than the first size. In some embodiments, the representationof the selected participant replaces the smallest representation in thecanvas region (corresponding to the participant in the canvas regionwith the lowest activity level), and then is automatically increased insize to (e.g., to the medium representation size) while anotherrepresentation (corresponding to the participant in the canvas regionwith the second lowest activity level) is reduced in size (e.g., to thesmall representation size).

Replacing the display of a representation corresponding to the selectedrepresentation in the overflow region and/or ceasing to display theselected representation in the overflow region provides additionalcontrol options without cluttering the UI and provides the user withmore control of the device and the display of representations by helpingthe user to quickly replace a representation with the user's selectedpresentation. Providing additional control of the device withoutcluttering the UI with additional displayed controls enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at block 914, the device displays a first animated effect(e.g., FIGS. 8AA-8AC) (e.g., the representation gets bigger, thensmaller, optionally with a crossfade and tint to create a “flash”effect) affecting the representation in the representations of the firstplurality of the participants. Optionally, at block 915, the devicedisplays a second animated effect affecting the selected representationin the representations of the second plurality of participants thatshares an animation characteristic with the first animated effect (e.g.,FIGS. 8BF-8BH) (e.g., a rate of fading/shrinking from the main displaycan correspond to a rate of flashing in the overflow display).Displaying the second animation that corresponds to the first animationprovides visual feedback to quickly identify which representations arebeing switched and/or replaced. Providing improved visual feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 916, further, in response to detecting the input(e.g., 850H) corresponding to the overflow region, the device displays anew (e.g., a previously hidden) representation of the second pluralityof participants in the overflow region (e.g., 822A in FIGS. 8AG-8AH).Displaying a new representation of a participant in the overflow regionin response to input allows the user to view representations easier andmore efficiently by increasing the accessibility of displayedrepresentations. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at block 917, in response to receiving data indicating thata new participant has joined or been added to the live communicationsession, the device displays a representation of the new participant inthe overflow region at the end of a sequence of the representations ofthe second plurality of participants in the overflow region (e.g., 822Bin FIG. 8R).

Optionally, at blocks 918, 919, 920, and 921, the device receives seconddata related to an activity (e.g., speaking, moving) level of a firstparticipant in the second plurality of participants. In response toreceiving the second data, the device, in accordance with the activitylevel of the first participant in the second plurality of participantsincreasing, emphasizes (e.g., increase or initiate) a visualcharacteristic (e.g., size, highlight representation (e.g., displayborder), animation (e.g., pulsing avatar), bring image to front) of arepresentation that corresponds to the representation of the firstparticipant in the representations of the second plurality ofparticipants in the overflow region (e.g., 822B of FIG. 8U). In responseto receiving the second data, the device, in accordance with theactivity level of the first participant in the second plurality ofparticipants decreasing, deemphasizes (e.g., increase or initiate) avisual characteristic (e.g., size, highlight representation (e.g.,display border), animation (e.g., pulsing avatar), bring image to front)of the representation that corresponds to the first participant in therepresentations of the second plurality of participants in the overflowregion. Emphasizing and/or deemphasizing a visual indication whenprescribed conditions are met allow the user to quickly recognize theactivity level of participants including participants that are mostand/or least active. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently. In addition, emphasizing and/or deemphasizing whenprescribed conditions are met provides visual feedback to the userindicating the activity level of each participant. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 922 and 923, while displaying the livecommunication user interface, the device detects a second participant inthe second plurality of participants becoming a currently activeparticipant (e.g., a most active participant, a participant that isspeaking or moving in a manner that satisfies an active participant setof criteria). In response to detecting the second participant in thesecond plurality of participants becoming the currently activeparticipant and in accordance with a determination that a representationof the second participant in the second plurality of participants is notcurrently displayed in the overflow region, the device scrolls theoverflow region to display the representation of the second participantin the second plurality of participants in the overflow region (e.g.,822F of FIGS. 8BH-8BI). Dynamically scrolling the overflow region todisplay an active participant in the overflow region allows viewing ofthe representation of the active participant in the overflow regionwithout selecting any additional inputs. Reducing the number of inputsneeded to perform an operation enhances the operability of the deviceand makes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

At block 924, in accordance with a determination that the number ofparticipants in the live communication session includes less than orequal to the threshold number of participants, the device displaysrepresentations of the participants other than the user of theelectronic device at one or more sizes greater than a respectivethreshold size without displaying representations of participants otherthan the user of the electronic device at one or more sizes smaller thanor equal to the respective threshold size (e.g., FIG. 8P). In someembodiments, the electronic device receives (e.g., prior to displayingthe user interface for the live communication session) a plurality oflive media streams associated with respective participants of aplurality of participants of a live video communication session. In someembodiments, the number of received live media streams corresponds tothe number of participants currently joined in the live videocommunication session, which does not include the participant using thedevice. Displaying representations of participants differently based onwhether prescribed condition are met (e.g., based on whether the livecommunication session includes less than or equal to a number ofparticipants display representations without an overflow region) allowsthe user to view representations of participants in the livecommunication session easier or more efficiently. Performing anoptimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, in accordance with a determination that the numberof participants in the live communication session includes less than asecond threshold number of participants, none of the plurality ofrepresentations of remote participants (e.g., participants other thanthe user) overlap each other (e.g., FIG. 8J). In some embodiments, inaccordance with a determination that the number of participants in thelive communication session includes more than or equal to the secondthreshold number of participants, at least some of the plurality ofrepresentations of the remote participants (e.g., participants otherthan the user) overlap each other (e.g., FIG. 8K).

Optionally, at blocks 925 and 926, in accordance with a determinationthat the number of participants in the live communication sessionincludes less than a second threshold (e.g., two participants, includingthe user of the electronic device (e.g., one other participant)) numberof participants, the device displays at least one of the representationsof the participants other than the user of the device at a first aspectratio (e.g., 810B of FIG. 8H) (e.g., a rectangular aspect ratio). Inaccordance with a determination that the number of participants in thelive communication session includes more than the second threshold(e.g., three or more participants) number of participants, the devicedisplays at least one of the representations of the participants otherthan the user of the device at a second aspect ratio (e.g., a squareaspect ratio), different than the first aspect ratio (e.g., 810B of FIG.8I). Displaying representations of the participants at different aspectratios based on the number of participants in the live communicationsession based on prescribed conditions being met allows the device toprovide an intelligent live communication user interface by maximizingthe size of the representations and/or the organization of therepresentation to provide a better user experience based on the numberof participants, allowing the user to view representation on the deviceeasier and more efficiently. Performing an optimized operation when aset of conditions has been met without requiring further user inputenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 927 and 928, in accordance with a determinationthat the number of participants in the live communication sessionincludes less than a second threshold (e.g., two participants) number ofparticipants, the device displays a representation of a view of a cameraof the device (e.g., a self-view from a camera of the device) at a thirdaspect ratio (e.g., a rectangular aspect ratio) (e.g., 806 of FIG. 8H).In accordance with a determination that the number of participants inthe live communication session includes more than or equal to the secondthreshold (e.g., three or more participants) number of participants, thedevice displays the representation of the view of the camera of thedevice (e.g., a self-view from a camera of the device) at a fourthaspect ratio (e.g., a square aspect ratio), different than the thirdaspect ratio (e.g., 806 of FIG. 8J). Displaying a view of the camera ofthe device at different aspect ratios based on the number ofparticipants in the live communication session based on prescribedconditions being met allows the device to provide an intelligent livecommunication user interface by maximizing the size of therepresentation of the view of the camera to the user and allowing theuser to view a representation on the device easier and more efficiently.Performing an optimized operation when a set of conditions has been metwithout requiring further user input enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 929 and 930, in accordance with the number ofparticipants in the live communication session being a first number ofparticipants, the device displays the representations of theparticipants in a first layout (e.g., arrangement or pattern) (e.g.,FIG. 8J). In some embodiments, the first layout is one of a plurality ofpredefined layouts in which representations of users are anchored atdifferent locations in the user interface (e.g., layouts selectedrandomly). In some embodiments, the layout (e.g., arrangement) of therepresentations of a first plurality of the participants (e.g., otherthan a user of the device) is determined based on the number ofparticipants in the live communication session. In accordance with thenumber of participants in the live communication session being a secondnumber of participants, the device displays the representations of theparticipants in a second layout, different than the second layout (e.g.,FIG. 8K) (e.g., layout changes as participants join/leave the livecommunication sessions). In some embodiments, the first layout and thesecond layout are in a family of predetermined layouts that have similarrelative placement of representations of users. (e.g., a family ofpredetermined layouts includes layouts that work well together). In someembodiments, a family of predetermined layouts includes layout membersfor a plurality of possible participant numbers (e.g., a first memberfor one non-user of the device participants (e.g., a 1-up layout), asecond member for two non-user-of-the-device participants (e.g., a 2-uplayout), and so forth. In some embodiments, the electronic deviceincludes a plurality of families of predetermined layouts, where themembers of each family share a common characteristic). Displayingrepresentations in different layouts that have similar relativeplacement of representations of users based on the number of usersallows the user view the representations easier and more efficientlywith increased visibility. Performing an optimized operation when a setof conditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 931, 932, 923 and 934, while displayingrepresentations of the participants other than the user of the device,the device receives data indicating that the number of participants inthe live communication session has increased. In response to receivingthe data indicating that the number of participants in the livecommunication session has increased, the device decreases the size ofone or more of the representations of participants other than the userof the device (e.g., FIGS. 8J-8K). In response to receiving the dataindicating that the number of participants in the live communicationsession has increased, the device displays one or more representationsof new (e.g., participants who have recently just joined the groupbefore the original display) participants in the live communicationsession (e.g., FIGS. 8J-8K).

Decreasing the size of one or more representations and displaying a newrepresentation provides the user with feedback about the current stateof the live communication session and provides visual feedback to theuser indicating that a new participant has joined the livecommunication. Providing improved visual feedback to the user enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 935 and 936, the device receives an input (e.g.,850B) corresponding to selection of the first reminder affordance (e.g.,808). In response to receiving the input corresponding to selection ofthe first reminder affordance, the devices sends a new notification(e.g., call, ring, etc.) to the first participant that is different thana previous notification sent to the first participant (e.g., the secondnotification is more intrusive than the first notification). Sending anotification to the first participant in response to receiving an inputcorresponding to a selection of a reminder affordance that is differentfrom a previous notification sent to the participant reduces the numberof inputs needed to perform an operation. Reducing the number of inputsneeded to perform an operation (e.g., the number of options needed forthe user to remind the participant) enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, performingan operation automatically, such as sending a notification that isdifferent from a first notification, without further user input)enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In embodiments where the placeholder and the representations of thefirst plurality of the participants are arranged in a first layout,optionally, at blocks 937, 938, and 939, the device, in accordance witha determination that the first participant has not joined the livecommunication session for a predetermined amount of time (e.g., notjoined the live communication after a predetermined period of time),ceases to display the placeholder (e.g., FIGS. 8E and 8F). Further, inaccordance with a determination that the first participant has notjoined the live communication session for a predetermined amount oftime, the device changes the arrangement of the representations of thefirst plurality of the participants to a second layout, different thanthe first layout (e.g., includes enlarging and/or moving one or more ofthe representations of participants (e.g., display a layout with thefirst participant not included in the layout). In some embodiments, thedevice receives data from a server or another device to make thedetermination that the first participant has not joined the livecommunication session for a predetermined amount of time. In someembodiments, the device can make the determination that the firstparticipant has not joined the live communication session for apredetermined amount of time because it hasn't displayed anyrepresentations of the user. Ceasing to display the placeholder andchanging the arrangement of the representations when prescribedconditions are met allows users to quickly recognize when other usersare joining or not joining the call while maximizing the visibility ofdisplayed participants and/or placeholders. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, ceasing todisplay the placeholder and changing the arrangement of the displayprovides the user with feedback about the current state of a livecommunication session, and provides visual feedback to the userindicating that certain operations associated with the notificationand/or the live communication session will be performed. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 940, in accordance with a determination that thefirst participant has joined the live communication session, the devicesmodifies the display of the first placeholder with a representation ofthe first participant (e.g., 810B of FIGS. 8C-8D) (e.g., a placeholdercan have participants name). In some embodiments, the device can receivedata from a server or another device to make this determination. In someembodiments, the device can make this determination because it hasn'tdisplayed any representations of the user. In some embodiments, ananimation occurs. Modifying the display of the first placeholder with arepresentation of the participant allows the user to quickly recognizewhich users have joined the call, provides the user with feedback aboutthe current state of a live communication session, and provides visualfeedback to the user indicating that certain operations associated withthe notification and/or the live communication session will beperformed. Providing improved visual feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device),which additionally, reduces power usage and improves battery life of thedevice by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 941 and 942, while displaying the plurality ofrepresentations of participants in the live communication session, thedevice detects a change in the number of participants in the livecommunication session. In response to detecting the change in the numberof participants in the live communication session, displaying anotification (e.g., 812) indicating that the number of participants inthe live communication session has changed. In some embodiments, thenotification, when a participant leaves the live communication session,is an animation of the departing participant's representation beingreplaced with a placeholder graphic (e.g., a graphic that includes theparticipant's name or initials) that subsequently disappears (e.g.,after a predetermined time). In some embodiments, the notification, whena new participant joins the live communication session, is display of aplaceholder (e.g., a graphic that includes the participant's name orinitials) for the new participant that is subsequently replaced (e.g.,after a predetermined time) with a representation (e.g., arepresentation based on video data transmitted from a device of the newparticipant) of the new participant. In some embodiments, thenotification indicating that the number of participants in the livecommunication session has changed is displayed in accordance with adetermination that the number of participants joined in the livecommunication session exceeds a threshold number of participants (e.g.,when the overflow region includes more than a predefined maximum numberof representations (e.g., 3 or 4 representations other than the user ofthe electronic device). In some embodiments, the device forgoesdisplaying a notification that the number of participants in the livecommunication session has changed (e.g., the notification is notdisplayed when there is another visible indication that a participanthas joined or left the live communication session, such as display of arepresentation of a newly joined participant or removal of the displayof a representation of a participant who has left the session). In someembodiments, the device provides an audio output indicating that thenumber of participants in the live communication session has changedwithout displaying a notification (e.g., when there is no visibleindication that a participant has joined or left the live communicationsession). In some embodiments, the notification includes an indication(e.g., the name) of a participant that has joined or left the livecommunication session. In some embodiments, if multiple participantsjoin and/or leave the live communication session within a thresholdamount of time, the device generates a single notification thatindicates the number of participants that joined and/or left the livecommunication session within the threshold amount of time. In someembodiments, participants joining the live communication session andparticipants leaving the live communication session are grouped togetherinto separate notifications. In some embodiments, when a participantleaves or joins the live communication session, the device delaysdisplaying a notification until the threshold amount of time has elapsedin order to determine whether the notification should includeinformation about additional participants joining and/or leaving thelive communication session. Displaying a notification indicating thatthe number of participants in the live communication session has changedprovides the user with feedback about the current state of the number ofparticipants in the live communication and provides visual feedback tothe user indicating that a change has occurred in the number ofparticipants in the live communication session. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 943, 944, 945, and 946, the device displays, aspart of the plurality of representations of participants in the livecommunication session, a representation (e.g., self-view) of a view of acamera of the device at a first size (e.g., 806 of FIG. 8AH). Whiledisplaying the plurality of representations of participants in the livecommunication session, the device receives a first input (e.g., 850I)(e.g., triggered based on single tap on background or up swipe)corresponding to selection for displaying a first menu (e.g., 832). Inresponse to receiving the input corresponding to selection fordisplaying the first menu, the device displays the first menu (e.g.,call control menu 832) in the live communication user interface (e.g.,804), where the first menu includes an option (e.g., 836) for displayinga visual effect (e.g., creative camera) on a representation of aparticipant in the live communication session. In some embodiments, menuincludes other options, such as to end the call switch cameras (e.g.,front-facing camera to back facing camera). In response to receiving asecond input (e.g., 850K) corresponding to selection of the option fordisplaying a visual effect (e.g., 842), the device displays (e.g.,overlaying on representations of participants) the representation of theview of the camera of the device at a second size (e.g., 842), whereinthe second size is greater than the first size.

Optionally, at blocks 947, 948, and 949, the device receives a thirdinput (e.g., 850C) (e.g., double tap on representation in the mainregion or an input having a characteristic intensity that exceeds anintensity threshold) corresponding to selection for enlarging a firstrepresentation in the plurality of representations of the participantsin the live communication session (e.g., 810F of FIG. 8L). In responseto receiving the third input corresponding to selection for enlargingthe first representation in the plurality of representations of theparticipants in the live communication session, the device enlarges thefirst representation (e.g., 810F of FIG. 8M). The device reduces a sizeof at least one representation in the plurality of representations ofthe participants in the live communication user interface that isdifferent than the first representation (e.g., 810E of FIGS. 8L-8M).Dynamically reducing the sizing of one or more representations in a setof displayed representations and enlarging a selected representationprovides improved feedback that allows a user to focus and easily viewthe enlarged representation while minimizing the reducedrepresentations. Providing improved visual feedback to the user enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 950, 951, 952, and 953, the device receives dataindicating a change in orientation of the electronic device (e.g., in afirst direction (e.g., change of 90 degrees clock)). In response toreceiving the data indicating a change in orientation of the electronicdevice, the device rotates the display of the plurality ofrepresentations of the participants in the live communication session(e.g., FIGS. 8W-8X) (e.g., in a second direction (e.g., opposite thefirst direction to maintain that the images are upright)). The device,while rotating the display, maintains relative properties (e.g., size,position of user interface objects (e.g., representations),arrangements, etc. of another display) of the previous display of theplurality of representations of the participants in the livecommunication session. Automatically rotating the device and maintainingrelative properties of previously displayed representations when certainconditions are met (e.g., data indicating that the user is rotating thedevice) allows the user to view the rotated representation in the samepreviously displayed positions without changing the arrangement, size,or positions of the representation with minimal visual distractions.Performing an optimized operation when a set of conditions has been metwithout requiring further user input enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 954, in response to receiving first data indicatinga change in orientation of the electronic device (e.g., from portraitorientation to landscape orientation), the device changes the display ofa first representation in the plurality of representations of theparticipants in the live communication session based on a position of aface in the first representation (e.g., 822C of FIGS. 8AZ-8BA) (e.g., ifthe representations are not square, adjust the display so that theparticipant's face is in the center of the representation when goingfrom a portrait representation (where the participant's face might bebiased toward the top of the representation) to a landscaperepresentation.).

Optionally, at blocks 955 and 956, while displaying the secondrepresentation of the plurality of representations of participants, thedevice receives a request to change the aspect ratio of at least onerepresentation of the plurality of representations of participants inthe live communication session. In response to receiving the request tochange the aspect ratio of at least one representation of the pluralityof representations of participants in the live communication session,the device changes the aspect ratio of the second representation fromthe sixth aspect ratio to a seventh aspect ratio (e.g., a square orcircular aspect ratio), where the second representation, while at theseventh aspect ratio, includes a second portion (e.g., a portion that isless than the first portion, a cropped portion) of the live media streamreceived by the electronic device, the second portion selected toinclude a face detected in the live media stream (e.g., 822C of FIGS.8AZ-8BA). In some embodiments, the electronic device changes the aspectratio of a representation based on a live video feed and crops a portionof the live video feed by selecting a portion of the live video feedthat includes a detected face so that the detected face is not croppedout of the representation. Maintaining a view of the faces inrepresentation of participants provides the user with feedback allowingthe user to view the other participants' faces when changing the aspectratio. Providing improved visual feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 957, 958, 959, 960, and 961, while displaying theplurality of representations of participants in the live communicationsession, the device receives a fourth input (e.g., 850I) (e.g.,triggered based on single tap on background or up swipe) correspondingto a request to display a menu (e.g., 832). In response to receiving thefourth input, device displays in the user interface for the livecommunication session, a menu (e.g., 832). The device moves at least athird representation of the plurality of representations of participantsin the live communication session (e.g., 810H of FIG. 8AI) (e.g., inorder to accommodate the menu; in order to avoid overlapping with themenu). The device changes the size of at least a fourth representation(e.g., a representation that is the same as the third representation; arepresentation that is different than the third representation) of theplurality of representations of participants in the live communicationsession (e.g., 810F of FIG. 8AI) (e.g., in order to accommodate themenu; in order to avoid overlapping with the menu). In some embodiments,representations are moved up and resized when a menu is displayed at thebottom of the screen. In some embodiments, at least one representationof the plurality of representations is not re-sized when displaying themenu. In some embodiments, the representations are representationshaving three predefined image sizes (e.g., 1 “small”, 2 “medium”, and 1“large”), and only representations of one size (e.g., “medium” image(s))are re-sized while keeping “small” and “large” representations the samesize). Moving one or more representations and changing the size of oneor more representations in the live communication session whendisplaying a menu provides additional control options while maintainingthe visibility of the representations in the live communication userinterface. Providing additional control options without cluttering thelive communication user interface with additional displayed controlsenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, the device receives data indicating an increase inan activity level of a first participant in the second plurality ofparticipants represented in the overflow region above an activity levelof a participant corresponding to a first representation in therepresentations of the first plurality of participants (e.g., aparticipant represented in the overflow region begins talking such thathis activity level increases above the activity level of one of theparticipants represented in the canvas region), and in response:replaces the display of the first representation in the representationsof the first plurality of the participants with a representation of thefirst participant in the second plurality of participants (e.g.,replaces the participant in the canvas region having the lowest activitylevel with the representation of the speaking participant); and ceasesto display the representation of the first participant in the overflowregion. In some embodiments, ceasing to display the representation ofthe first participant in the overflow region includes an animation ofthe representation shrinking. In some embodiments, other representationsin the overflow region are translated to fill a gap where therepresentation was displayed. In some embodiments, replacing the displayof the first representation in the representations of the firstplurality of the participants includes displaying, in the overflowregion, a representation of the participant corresponding to thereplaced first representation.

In some embodiments, while displaying the plurality of representationsof participants in the live communication session, the device receivesan input corresponding to a request to display a menu. In response toreceiving the input corresponding to the request to display the menu,the device displays the menu in the live communication user interface,where the menu includes an option for selecting a content item (e.g.,game, video, sketch). In some embodiments, the device receives an inputcorresponding to selection of a content item, and in response toreceiving the input corresponding to selection of the content item, thedevice replaces a representation of a participant displayed at a sizegreater than the respective threshold size (e.g., a representation of aparticipant in the canvas region) with a representation of the contentitem at a size greater than the respective threshold size (e.g., thecontent item is displayed in one of the positions in the canvas region).

In some embodiments, displaying representations of a second plurality ofthe participants in an overflow region includes: in accordance with adetermination that a first number of participants (e.g., 6 totalparticipants) are participating in the live communication session,displaying a first set of representations in the overflow region (e.g.,two representations, including a representation of the least activeparticipant and the representation of the view of the camera of thedevice); and in accordance with a determination that a second number ofparticipants (e.g., 7 total participants) different than the firstnumber of participants are participating in the live communicationsession, displaying a second set of representations (e.g., threerepresentations, including representations of the two least activeparticipant and the representation of the view of the camera of thedevice) different than the first set of representations in the overflowregion, where the second set of representations includes a differentnumber of representations than the first set of representations.

Note that details of the processes described above with respect tomethod 900 (e.g., FIG. 9A-9K) are also applicable in an analogous mannerto the methods described below/above. For example, method 900 optionallyincludes one or more of the characteristics of the various methodsdescribed above with reference to methods 700, 1100, 1300, and 1500.

FIGS. 10A-10S illustrate exemplary techniques for, inter alia,dynamically adjusting user interfaces of live communication sessions, inaccordance with some embodiments. The user interfaces in these figuresare used to illustrate the processes described below, including theprocesses in FIG. 11A-11F.

FIG. 10A illustrates device 600 displaying user interface 1000 of anactive live communication session. User interface 1000 includesrepresentations of five participants in the live communication sessiondisplayed in canvas region 1001. Representation 1010A includes aplaceholder for a participant that has not connected to the livecommunication session. Representation 1010B includes a representativeimage for a participant that is providing only audio data.Representations 1010C and 1010D include live video feeds from tworespective participants. Representation 1002 includes a view from camera603 of device 600.

Each participant has an activity level that is used to determine thelayout of user interface 1000. In some embodiments, the activity levelof a participant is based on an audio and/or video feed (e.g., motion orsound in the video feed) received from a device of the participant. Insome embodiments, the activity level of a participant is based onfiltered audio (e.g., audio filtered of background noise or filtered toisolate/amplify speech audio) from an audio feed received from thedevice of the participant. In some embodiments, the activity level of aparticipant is based on movement identified in a video feed receivedfrom the device of the participant that meets a set of movement criteria(e.g., movement that is of a certain type (e.g., hand movement,head/face movement, movement that is not background movement (e.g.,non-participant movement)).

In some embodiments, the activity level of a participant is based ondata indicating activity detected at the device of the first participant(e.g., data indicating that the first participant performed a userinterface action at the device of the participant (e.g., selection of anattention affordance or a visual effect affordance). In someembodiments, the activity level is determined at an external electronicdevice (e.g., a server) and then transmitted to device 600.

As illustrated in FIG. 10A, the activity level of the participants oflive communication session is in the following order, from highest tolowest, by representation: representation 1010C, representation 1010D,representation 1010B, and representation 1010A. The tables shown on theright side of FIGS. 10A-10O list the participants in descending order ofactivity level from highest to lowest, with the participant at the topof the list (1) having the highest activity level and the person at thebottom of the list (4) having the lowest activity level of theparticipants represented in canvas region 1001. In FIGS. 10A-10O, therepresentation corresponding to the participant with the highestactivity level is the largest of the representations in canvas region1001, and the representation corresponding to the participant with thelowest activity level of the participants represented in canvas region1001 is the smallest. In FIGS. 10A-10O, the representationscorresponding to the participants with the second highest activity leveland third highest activity level have a size between the size of therepresentation of the participant with the highest activity level andthe size of the representation of the participant with the lowestactivity level of the participants represented in canvas region 1001.

In some embodiments, the activity level of a participant changesrelative to the other participants. While displaying user interface1000, device 600 receives data indicating that an activity level ofparticipant Stephen corresponding to representation 1010B has increasedabove the activity level of participant Nicholas corresponding torepresentation 1010D. As illustrated in FIG. 10B, the activity level ofrepresentation 1010B increases above the activity level ofrepresentation 1010D, but does not changes size since it has the secondto highest activity level. In FIG. 10B, participant Stephen continues tospeak and becomes the participant with the highest activity level. Inresponse, representation 1010B increases in size, representation 1010Cdecreases in size, and representation 1010A shifts to the right (e.g.,to avoid excessive overlap with representation 1010B due to theincreased size of representation 1002B), as illustrated in FIG. 10C. Insome embodiments, device 600 changes the size and/or position of threeor more representations on user interface 1000.

As illustrated in FIG. 10C, representations 1010A-1010D overlap witheach other. In some embodiments, one or more of representations1010A-1010D are reduced in size and/or moved when a representation isbeing increased in size in order to avoid obstructing a face of aparticipant and/or avoid creating an overlap that exceeds a predefinedmaximum overlap. In some embodiments, the front-to-back (aka “z-order”)of representations 1010A-1010D does not change when a representation isincreased in size (e.g., when a participant is actively participating).In some embodiments, a representation 1010 changes order (e.g., movesfrom behind to in front or from in front to behind) one or morerepresentations when changing size (e.g., due to active participation bythe corresponding participant or a user input at device 600 selecting arepresentation (e.g., in response to user input 850C in FIG. 8Mdescribed above)). For example, in some embodiments, representation1010B is moved in front of representation 1010C and/or representation1010A when increasing in size in FIGS. 10B-10C.

In some embodiments, the representations of the participants remain inthe same general region on display 602 when changing size and/orposition. As illustrated in FIGS. 10A-10C, after representations1010A-1010D change size and/or position, each representation occupies atleast part of the area it occupied prior to changing size and/orposition.

Turning to FIG. 10D, participant Stephen of representation 1010B beginsto stop talking, and, as illustrated in FIG. 10E, the participant in thelive video feed of representation 1010A begins to physically move. Theparticipant's motion is detected in the live video feed. In someembodiments, a participant is determined to be actively participating inthe live communication session based on motion detected in a live videofeed of the participant (e.g., motion indicative of physical movement ofthe participant). In some embodiments, movement of a participant resultsin an increase in the activity level of the participant. In someembodiments, a participant is determined to be actively participating inaccordance with the motion meeting one or more criteria (e.g., duration,magnitude, direction, or predetermined gesture).

As illustrated in FIG. 10F, in response to receiving data indicatingthat the participant of representation 1010A is actively participating,device 600 increases the size of representation 1010A and decreases thesize of representation 1010D. Representation 1010B remains the same sizeand is moved downward. In this way, representation 1010A transitionsfrom the smallest representation to a medium size representation andrepresentation 1010D transitions to being the smallest representation.

In some embodiments, in response to receiving data indicating that theparticipant of representation 1010A (e.g., the participant having thelowest activity level) is actively participating, device 600 increasesthe size of representation 1010A from the smallest representation to thelargest representation, and decreases the size of representation 1010B.In some embodiments, representation 1010A is increased from the smallestrepresentation to the largest representation in accordance with theparticipant corresponding to representation 1010A being the most recentparticipant to actively participate. For example, in some embodiments,in accordance with the participant corresponding to representation 1010Abeing the most recent participant to actively participate, device 600transitions from the layout and relative representation sizesillustrated in FIG. 10E to the layout and relative representation sizesillustrated in FIG. 10G (discussed below).

In some embodiments, a participant actively participates in the livecommunication session by applying a visual effect to the live video feedprovided to the live communication session. In some embodiments, aparticipant applies a visual effect via a user interface of theparticipant's device. In some embodiments, a participant is determinedto be actively participating in the live communication session based ona visual effect included in the live media feed of the participant. Asillustrated in FIG. 10F, a visual effect (robot face) is applied to thelive video feed of representation 1010A. As a result, the participant ofrepresentation 1010A is determined to be actively participating and theparticipant's activity level is increased. As illustrated in FIGS.10F-10H, in accordance with the determination that the participant ofrepresentation 1010A is actively participating, device 600 increases thesize of representation 1010A to the largest representation size andreduces the size of representation 1010B to a medium representationsize. The relative z-order of representations 1010A and 101B alsochange, as representation 1010A moves in front of representation 1010Bdue to the increased activity level of the participant associated withrepresentation 1010A.

In FIG. 10H, device 600 receives (e.g., detects) user input 1050A (e.g.,a tap) on representation 1010C. As illustrated in FIG. 10I-10J, inresponse to user input 1050A, device 600 increases the size ofrepresentation 1010C to the largest representation size. In addition,the participant corresponding to representation 1010D begins activelyparticipating (e.g., talking), causing representation 1010D to increaseto a medium size and representation 1010B to reduce to a small size.

In FIG. 10J, device 600 receives (e.g., detects) user input 1050B (e.g.,a tap) on a location of display 602 at which a representation of aparticipant is not displayed. As illustrated in FIG. 10K, in response touser input 1050B, device 600 displays call control menu 1004, which isanalogous to call control menu 630 described above.

As illustrated in FIG. 10K, the size of canvas region 1001 is reducedwhen call control menu 1004 is displayed such that call control menu1004 does not overlap with the representations 1010. To make room forcall control menu 1004, device 600 modifies the layout ofrepresentations 1010 by reducing the size and/or vertically translatingupward one or more of the representations. In some embodiments, device600 reduces the size of the medium sized representations, 1010A and1010D, and maintains the size of the small and large primaryrepresentations, 1010B and 1010C, respectively. In some embodiments,device 600 reduces the vertical separation between representations(e.g., between representations 1010A and 1010C).

In response to receiving user input 1050B, device 600 also displaysribbons 1012A-1012D on each of the representations, except forrepresentation 1002 of the user of device 600. Each ribbon includes thename of the corresponding participant and an affordance to enlarge therepresentation.

Turning to FIG. 10L, device 600 is illustrated in a landscape(horizontal) orientation. The relative activity levels of theparticipants are the same as in FIG. 10J. As compared to the portrait(vertical) orientation in FIG. 10J, the representations occupy the samerespective areas of display 602 as in the portrait (vertical)orientation and images within the representations are rotated so thatthe participants appear upright. In some embodiments, in response torotation of device 600 from the portrait orientation of FIG. 10J to thelandscape orientation of FIG. 10L, device 600 displays an animatedrotation of the representations or the images within the representations(e.g., 90 degrees opposite the direction of rotation of device 600).

Turning to FIG. 10M, beginning from the layout and activity levelsillustrated in FIG. 10J, a sixth participant Pablo joins the livecommunication session. As illustrated in FIG. 10M, in response to thesixth participant joining the live communication session, and inaccordance with a determination that the number of participantsconnected to the live communication session is greater than apredetermined threshold of five connected participants, device 600replaces the image in representation 1010B corresponding to theparticipant with the lowest activity level with an image of the newlyconnected participant. Device 600 optionally provides a notificationthat the newly connected participant has joined the live communicationsession and displays an overflow representations 1022A of the newlyconnected participant in overflow region 1020. As illustrated in FIGS.10M-10O, the size of canvas region 1001 is reduced when overflow region1020 is displayed. At least some of representations 1010A-1010D arereduced in size and/or moved on display 602.

As illustrated in FIG. 10N, overflow representation 1010A corresponds toa participant that is not represented by a primary representation (e.g.,the representations above overflow region 1020 in FIG. 10N). In FIG.10N, device 600 receives (e.g., detects) user input 1050C (e.g., a tap)on overflow representation 1022A. As illustrated in FIG. 10O, inresponse to user input 1050C, device 600 replaces the image inrepresentation 1010A with an image of the participant corresponding toselected overflow representation 1022A and displays overflowrepresentation 1022B corresponding to the replaced participant (Hoan).

In some embodiments, device 600 replaces the image in representation1010A of FIG. 10N with an image of the participant corresponding tooverflow representation 1022A in response to active participation by theparticipant corresponding to overflow representation 1022A (e.g.,instead of in response to user input 1050C). In some embodiments, device600 replaces the image in representation 1010A of FIG. 10N with an imageof the participant corresponding to overflow representation 1022A inresponse to a determination that the activity level of the participantcorresponding to overflow representation 1022A has increased above theactivity level of the participant corresponding to representation 1010Aof FIG. 10N.

Turning to FIG. 10P, device 1008 with display 1014 is illustrated. Insome embodiments, display 1014 has different characteristics (e.g., sizeand/or aspect ratio) than display 602 of device 600. Device 1008displays user interface 1016, which is divided into two portions 1016A(e.g., the canvas region) and 1016B. Portion 1016A has approximately thesame aspect ratio as display 602 of device 600. In accordance with theaspect ratio of portion 1016A, portion 1016A includes representations1030A-1030D, corresponding to the participants of representations1010A-1010D in FIG. 10O. As illustrated in FIG. 10P, the layout (e.g.,relative size, positions, spacing, and overlapping) of representations1030A-1030D is similar to the layout of representations 1010A-1010D inFIG. 10O.

As illustrated in FIG. 10P, device 1008 receives (e.g., detects) userinput 1050D (e.g., a right swipe) to move divider 1018 to the right. Inresponse, device 1008 reduces the size of portion 1016B and increasesthe size of portion 1016A. As illustrated in FIGS. 10P-10S,representations 1030 are enlarged and the horizontal spacing betweenrepresentations 1030 is increased. As illustrated in FIG. 10S, device1008 displays menu 1024, with effects affordance 1026, call affordance1028, and menu affordance 1034 (which are analogous to effectsaffordance 632, call affordance 634, and menu affordance 636 describedabove).

FIG. 11A-11F is a flow diagram illustrating a method in accordance withsome embodiments. Method 1100 is performed at a device (e.g., 100, 300,500, or 600) with a display. Some operations in method 1100 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 1100 provides an intuitive way for providingan adaptive and intelligent live communication user interface thatemphasizes one or more active participants in the live communicationsessions. The method reduces the cognitive burden on a user for viewingactive participants in a live communication session, thereby creating amore efficient human-machine interface. For battery-operated computingdevices, enabling a user to view active participants faster and moreefficiently conserves power and increases the time between batterycharges.

At block 1102, the device displays a live video communication userinterface (e.g., 1000) that includes concurrently displaying arepresentation (e.g., 1010C) of a first participant in a live videocommunication session, a representation (e.g., 1010B) of a secondparticipant in the live video communication session, and arepresentation (e.g., 1010D) of a third participant in the live videocommunication session. In some embodiments, if a participant is sharinga live media stream that includes a live video stream, therepresentation includes images of the live video stream (e.g., 1010C).In some embodiments, if a participant is sharing a live media streamwith only audio, the representation includes an avatar of the respectiveparticipant (e.g., 1010B). In some embodiments, if a participant is notcurrently joined in the live video communication session, therepresentation includes a placeholder image (e.g., an avatar) (e.g.,1010A). In some embodiments, the live communication interface includes acanvas region (e.g., 1001) (e.g., canvas region (e.g., main region) isdifferent from the overflow region. In some embodiments, the canvasregion is next to the overflow region. In some embodiments, therepresentation of the first participant in the live communicationsession, the representation of the second participant in the livecommunication session, and the representation of the third participantin the live communication session are displayed in the canvas region(e.g., FIG. 10A).

At block 1104, while displaying the live video communication userinterface, the device receives data indicating that an activity level ofthe second participant (e.g., 1010B) in the live video communicationsession has increased above the activity level of the first participant(e.g., 1010C) in the live video communication session. In someembodiments, data indicating that the first participant is activelyparticipating is received from an external source (e.g., a server). Insome embodiments, the device determines that the first participant isactively participating is based on received data (e.g., live video orlive audio from a live media feed associated with the first participant.In some embodiments, prior to receiving the data indicating that theactivity level of the second participant in the live video communicationsession has increased above the activity level of the first participantin the live video communication session, the one or more of therepresentation of the first participant, the second participant, and thethird participant overlaps with at least one other representation of aparticipant (e.g., 1010B, 1010C, and 1010D of FIG. 10A).

In some embodiments, the activity level of at least the firstparticipant is based on (e.g., a function of) an audio and/or video feed(e.g., motion or sound in the video feed) received from a device of thefirst participant and/or data indicating activity detected at the deviceof the first participant (e.g., data indicating that the firstparticipant performed a user interface action at the device of the firstparticipant (e.g., selection of a attention affordance)). In someembodiments, the activity level is determined at an external electronicdevice (e.g., a server) and then transmitted to the electronic device.In some embodiments, the activity level of at least the firstparticipant is based on filtered audio (e.g., audio filtered ofbackground or filtered to isolate/amplify speech audio) from an audiofeed received from the device of the first participant. In someembodiments, the activity level of at least the first participant isbased on movement identified (e.g., identified at a server; identifiedat the electronic device) in a video feed received from the device ofthe first participant that meets a set of movement criteria (e.g., 1010Aof FIG. 10E) (e.g., movement that is of a certain type (e.g., handmovement, head/face movement, movement that is not background movement(e.g., non-user movement)).

At block 1106, in response to receiving the data indicating that theactivity level of the second participant (e.g., 1010B of FIG. 10A) inthe live video communication session has increased above the activitylevel of the first participant (e.g., 1010C of FIG. 10A) in the livevideo communication session, the device changes a size and/or positionof the representation of the first participant in the live videocommunication user interface (at block 1108), changes a size and/orposition of the representation of the second participant in the livevideo communication user interface (at block 1110), and changes a sizeand/or position of the representation (e.g., 1010D) of the thirdparticipant in the live video communication user interface (at block1112). Changing the size and/or positions of representations whenprescribed conditions are met allows the user to quickly recognize theactivity level of participants including participants that are mostand/or least active. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently. In addition, changing the size and/or positions ofrepresentations only when prescribed conditions are met reduces thenumber of inputs needed for a user to determine an active participant orfocus on an active participant. Reducing the number of inputs furtherenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, in response to receiving the data indicating thatthe activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session, the size ofthe representation (e.g., 1010C of FIG. 10A) of the first participant inthe live communication user interface is changed, the size of therepresentation (e.g., 1010B of FIG. 10A) of the second participant inthe live communication user interface is changed, and the size of therepresentation (e.g., 1010D of FIG. 10A) of the third participant in thelive communication user interface is changed. In some embodiments, ifthe least active participant become the most active participant (e.g.,based on activity score), the device increases the size of therepresentation of the most active participant, and decreases the size ofall other representations (e.g., FIGS. 10E-10F). In some embodiments, ifthe most active participant becomes the least active participant (e.g.,based on an activity score), the device decreases the size of therepresentation of the least active participant, and increase the size ofall other representations. In some embodiments, in response to receivingthe data indicating that the activity level of the second participant inthe live video communication session has increased above the activitylevel of the first participant in the live video communication session:the position of the representation of the first participant in the livecommunication user interface is changed; the position of therepresentation of the second participant in the live communication userinterface is changed; and the position of the representation of thethird participant in the live communication user interface is changed.In some embodiments, if least active participants become the most activeparticipant (e.g., based on activity score), swap all positions (e.g.,representation of most active participants moves up or over, and othersmove down or over). In some embodiments, if the most active participantbecause the least active participant (e.g., based on an activity score),the representation of least active participants moves down and/or over,and other representations move up and/or over.

In some embodiments, a portion of the representation (e.g., 1010C ofFIG. 10A) of a participant having (e.g., currently having) the highestactivity level of the first, second, and third participants isoverlapped (e.g., a portion of representation for most active isunderneath the other representations) by a portion of the representationof another participant of the first second, and third participants inthe live communication user interface. In some embodiments, therepresentation of the participant having the lowest activity level isdisplayed without being overlapped by the representations of any otherparticipants (e.g., because the representation of the participant havingthe lowest activity level is the smallest representation of therepresentations of participants and thus is kept on top ofrepresentations of other participants to avoid obscuring too much of therepresentation of the participant with the lowest activity level). Insome embodiments, the representation (e.g., 1010B of FIG. 10I) of theparticipant having the lowest activity level is displayed overlapping atleast a portion of the representation (e.g., 1010A of FIG. 10I) of theparticipant having the highest activity level. In some embodiments, aportion of the representation (e.g., 1010D of FIG. 10H) of a participanthaving (e.g., currently having) the lowest activity level of the first,second, and third participants overlaps (e.g., a portion forrepresentation of least active on underneath of the otherrepresentations) a portion of the representation (e.g., 1010C of FIG.10H) of another participant of the first, second, and third participantsin the live communication user interface.

Optionally, at block 1114, the device changes the size and/or positionof the representation of the first participant based on facialrecognition data obtained from (e.g., obtained by analyzing) a videofeed of the second participant (and/or a video feed of the thirdparticipant), where changing the size and/or position of therepresentation of the first participant does not result in obscuring theface of the second participant (and/or obscuring the face of the thirdparticipant). Changing the size and/or position of a representation of aparticipant in the live communication session without obscuring a faceof another participant provides the user with feedback about thecurrently activity level of a participant without hiding the faces ofother participants in the live communication session, thereby allowingthe user to view the other participants' faces when changing in size orposition of the participant whose activity level has changed. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1116, the device changes the size and/or positionof the representation (e.g., 1010B of FIG. 10A-10C) of the firstparticipant includes gradually increasing (e.g., gradually over a periodof time) the size of the representation from a first size to a secondsize. Changing the sizes and/or positions of representations providesthe user feedback about the current activity level of the participantswhile minimizing user interface changes. Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. Increasing the size of arepresentation gradually provides visual feedback to the user indicatingthat the activity level of the participant is changing when theiractivity level changes while minimizing distractions that an abruptchange in size causes to the user. Providing improved visual feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1118, the device detects a canvas size change event(e.g., 1050B; FIGS. 10J-10K) (e.g., rotation of the device; theintroduction of additional elements (e.g., one or more call affordancesfor contacting participants; the addition of representations ofparticipants displayed outside the canvas region) into the live videocommunication user interface; a user input corresponding to a request tochange the size of the canvas region). Optionally, at block 1120, thedevice, in response to detecting the canvas size change event, changes asize of the canvas region (e.g., 1001) (optionally, at block 1124). Inresponse to detecting the canvas size change event, the device modifiesa layout of the representations of the first, second, and thirdparticipants (e.g., 1010B, 1010C, and 1010D of FIG. 10K). In someembodiments, the layout of the representations is modified by shiftingthe positions of the representations and/or changing the sizes of therepresentations (e.g., so as to reduce the spacing between therepresentations, when the size of the canvas region is reduced).Intelligently modifying a layout based the size of the canvas regionchanges allows users to view the representation of the users withminimal changes the displayed representations when the canvas sizechanges, and thereby, reducing distractions to the user while allowingusers to view the displayed presentations. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1126, the device further, in response to detectingthe canvas size change event, changes the size of one or more of therepresentations of the first, second, and third participants (e.g.,1010B, 1010C, and 1010D of FIG. 10K). In some embodiments, prior toreceiving the data indicating that the activity level of the secondparticipant in the live video communication session has increased abovethe activity level of the first participant in the live videocommunication session, the representations of the first, second, andthird participants in the live communication user interface aredisplayed according to a z-order (e.g., 1010B, 1010C, and 1010D of FIG.10K) (e.g., a “z-order” refers to the order of the representations ofthe participants with respect to a virtual axis normal to the face ofthe display) (e.g., an axis coming out of the screen) (e.g., when thepositions of two representations overlap, the z-order of therepresentations determines which object is displayed in front of theother). At block 1128, the device maintains the z-order of therepresentations of the first, second, and third participants (e.g.,1010B, 1010C, and 1010D of FIG. 10K).

In some embodiments, after changing the size and/or position of therepresentation (e.g., 1010B of FIG. 10B) of the first participant in thelive communication user interface, the changed representation of thefirst participant occupies at least a portion of a region (e.g., generalarea, location, pixel on the display, etc.) that was occupied by therepresentation of the first participant in the live communication userinterface before changing at least one of the size or the position ofthe representation of the first participant In some embodiments, afterchanging the size, the participant is displayed in the same region ofthe display as the participant was previously in (e.g., at least aportion (e.g., a part, a pixel, a position, etc.) of the representationcontinues to contact a location on the display that it previouslycontacted before changing). In some embodiments, the representationsmaintain arrangement/general location of representations whileemphasizing active participant (e.g., representations shift as theyincrease/decrease in size but maintain position relative to otherimages.). In some embodiments, after changing the size and/or positionof the representation (e.g., 1010C of FIG. 10B) of the secondparticipant in the live communication user interface, the changedrepresentation of the second participant occupies at least a portion ofa region (e.g., general area, location, pixel on the display, etc.) thatwas occupied by the representation of the second participant in the livecommunication user interface before changing at least one of the size orthe position of the representation of the second participant In someembodiments, after changing the size, the participant is displayed inthe same region of the display as the participant was previously in(e.g., at least a portion (e.g., a part, a pixel, a position, etc.) ofthe representation continues to contact a location on the display thatit previously contacted before changing). In some embodiments,representation maintains arrangement/general location of representationswhile emphasizing active participant (e.g., representations shift asthey increase/decrease in size but maintain position relative to otherimages). In some embodiments, after changing the size and/or position ofthe representation (1010D of FIG. 10B) of the third participant in thelive communication user interface, the changed representation of thethird participant occupies at least a portion of a region (e.g., generalarea, location, pixel on the display, etc.) that was occupied (e.g.,before the change) by the representation of the third participant in thelive communication user interface before changing at least one of thesize or the position of the representation of the third participant Insome embodiments, after changing the size, the participant is displayedin the same region of the display as the participant was previously in(e.g., at least a portion (e.g., a part, a pixel, a position, etc.) ofthe representation continues to contact a location on the display thatit previously contacted before changing) In some embodiments, therepresentations maintain their arrangement and/or general locationswhile emphasizing active participant (e.g., representations shift asthey increase/decrease in size but maintain position relative to otherimages).

Displaying representations that occupy at least a portion of the regionthat they previously occupied after changing the size and/or positionsof the representations allows the user to recognize the activity levelof participants with minimal changes to the user interface, therebyincreasing the users' ability to focus on representations ofparticipants in the live communication session. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1130, the device displays a representation (e.g.,1010D of FIG. 10E) of a fourth participant in the live communicationuser interface. Optionally, in response to receiving the data indicatingthat the activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session, the devicechanges a size and/or position of the representation of the fourthparticipant in the live communication user interface (at block 1132). Insome embodiments, a fourth representation changes the size and/orposition when all others change size or position. In some embodiments, afourth representation changes (e.g., in response to receiving the firstdata indicating that the activity level of the second participant in thelive communication session has increased above the activity level of thefirst participant in the live communication session). In someembodiments, some representations, such as the fourth representations,may maintain their size and position even when other representations arechanging).

Optionally, at blocks 1134, 1136, 1138, 1140, and 1142, after changingthe size and/or position of the representation of the first participantin the live video communication user interface, the device receivessecond data indicating that an activity level of the first participantin the live video communication session has increased above the activitylevel of the second participant in the live video communication session.In response to receiving the second data indicating that an activitylevel of the first participant in the live video communication sessionhas increased above the activity level of the second participant in thelive video communication session, the device changes the size and/orposition of the representation of the first participant in the livevideo communication user interface. The device changes the size and/orposition of the representation of the second participant in the livevideo communication user interface. The device maintains the size andposition of the representation of the third participant in the livecommunication session. In some embodiments, in response to changes inactivity level of a participant, the representation of some of therepresentations displayed in the canvas (e.g., a main region) aremaintained, even though other representations change size or position(e.g., two representations may swap places based on a determination andother representations their size of position). In some embodiments, therepresentation that is maintained corresponds to a participant having anactivity level lower than the activity level of the participants thatcorrespond to the representations that are changed. Maintaining the sizeand/or positions of some representations while changing the size and/orrepresentations of other representations when prescribed conditions aremet allows the user to quickly recognize the activity level ofparticipants with minimal changes to the user interface, thereby,increasing the users' ability to focus on representation of participantsin the live communication session. Performing an optimized operationwhen a set of conditions has been met without requiring further userinput enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 1144 and 1146, the device, in response toreceiving the first data indicating that the activity level of thesecond participant in the live communication session has increased abovethe activity level of the first participant in the live communicationsession, emphasizes (e.g., increase or initiate) a visual indication(e.g., highlight representation (e.g., display border), animation (e.g.,pulsing avatar), bring image to front, increase size) of therepresentation of the second participant (e.g., when the secondparticipant is currently more active) in the live communication userinterface. In response to receiving the first data indicating that theactivity level of the second participant in the live communicationsession has increased above the activity level of the first participantin the live communication session, the device deemphasizes (e.g.,decrease or cease) a visual indication (e.g., highlight representation(e.g., display border), animation (e.g., pulsing avatar), bring image toback, decrease size) of the representation of the first participant(e.g., when the first participant is currently less active) in the livecommunication user interface. Emphasizing and/or deemphasizing a visualindication when prescribed conditions are met allows the user to quicklyrecognize the activity level of participants including participants thatare most and/or least active. Performing an optimized operation when aset of conditions has been met without requiring further user inputenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In addition, emphasizing and/or deemphasizing whenprescribed conditions are met provides visual feedback to the userindicating the activity level of each participant. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, prior to (e.g., before) the device receiving thedata indicating that the activity level of the second participant in thelive video communication session has increased above the activity levelof the first participant in the live video communication session: therepresentation of the first participant is displayed at a first size(e.g., a middle size in a set of 3 or more predefined sizes; the largestsize in the set) the representation of the second participant isdisplayed at a second size (e.g., the smallest size in a set of 3 ormore predefined sizes; a middle size in the set) smaller than the firstsize (e.g., small representation). Optionally, at block 1148, the deviceincreases the size of the representation of the second participant fromthe second size to the first size. Increasing the size and/or positionof a representation from a smaller size to a larger size provides visualfeedback to the user indicating that the activity level of theparticipant is changing when their activity level changes. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 1150 and 1152, after receiving the data indicatingthat the activity level of the second participant in the live videocommunication session has increased above the activity level of thefirst participant in the live video communication session, the devicedetects an input (e.g., 850C) (e.g., tap on a representation)corresponding to the representation of the first participant. Inresponse to detecting the input corresponding to the representation ofthe first participant (e.g., a user can change the size of anyrepresentation displayed (e.g., in an overflow region, canvas region,etc.) on the live communication user interface), the device increasesthe activity level (e.g., increasing the value of an activity level) ofthe first participant (e.g., 810F of FIGS. 8L-8M).

At blocks 1154 and 1156, the device receives data indicating that afourth participant in the live communication session is more active thana participant having the lowest activity level of the first, second, andthird participants. In response to receiving the data indicating thatthe fourth participant in the live communication session is more activethan a participant having the lowest activity level of the first,second, and third participants, replace the display of therepresentation (e.g., 1010A of FIG. 10N) of the participant having thelowest activity level of the first, second, and third participants withdisplay of a representation (e.g., 1010A of FIG. 10O) of the fourthparticipant. In some embodiments, at block 1158, the device displays therepresentation of the fourth participant at a size larger than the sizeof the representation of the participant having the lowest activitylevel of the first, second, and third participants. Replacing thedisplay of the representation of a participant with the lowest activitylevel with a participant that is not displayed in the first set ofrepresentations when prescribed conditions are met allows the user tofocus on representations of the most active participants in the livecommunication session while decreasing the user's focus on the leastactive participants in the live communication session when users notdisplayed are more active than users who are displayed in the first setof representations. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently. Increasing the size of a newly displayed participantprovides the user feedback about the current activity level of the otherdisplayed participants (e.g., in the first subset of representations) inrelation to the new participant while minimizing user interface changes.Providing improved visual feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1160, the device displays a representation (e.g.,1002) of a view of a camera of the electronic device (e.g., (e.g.,self-view) in the live communication session user interface. Optionally,at block 1162, the device, further in response to receiving the firstdata indicating that the activity level of the second participant in thelive communication session has increased above the activity level of thefirst participant in the live communication session: optionally, atblock 1164, maintains a size of the representation (e.g., 1002) of aview of a camera of the electronic device in the live communicationsession user interface. In some embodiments, self-view does not changebased on changes in activity level of other participants). Maintainingthe size of a representation of a view of the camera of the device whileother representations may change when a set of conditions has been metallows users to view their representations while minimizing changes tothe layout continuously. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Note that details of the processes described above with respect tomethod 1100 (e.g., FIG. 11A-11F) are also applicable in an analogousmanner to the methods described below/above. For example, method 1100optionally includes one or more of the characteristics of the variousmethods described above with reference to method 700, 900, 1300, or1500.

FIGS. 12A-12N illustrate exemplary user interfaces for, inter alia,initiating a live communication session from a messaging application, inaccordance with some embodiments. The user interfaces in these figuresare used to illustrate the processes described below, including theprocesses in FIG. 13A-13D.

FIG. 12A illustrates device 600 (as described above) displayingmessaging user interface 1200 of a messaging application. Messaging userinterface 1200 includes messages in message region 1202 betweenparticipants in a group of 15 participants (“The Dream Team”) engaged ina message conversation. Messaging user interface 1200 includes call-allaffordance 1206 for initiating a live communication session with all ofthe participants in the message conversation (e.g., all of theparticipants in The Dream Team group) and group contact informationaffordance 1208 for accessing additional information related to TheDream Team group.

As illustrated in FIG. 12A, device 600 receives (e.g., detects) userinput 1250A (e.g., a tap) corresponding to selection of call-allaffordance 1206. In response to receiving user input 1250A, device 600starts a live communication session with all of the participants in themessage conversation included as participants in the live communicationsession. In some embodiments, in response to receiving user input 1250A,device 600 sends instructions causing devices associated with themembers of the group to display a notification that the livecommunication session is active. In this way, call-all affordance 1206immediately starts a live communication session with all of theparticipants in the message conversation group.

FIG. 12B illustrates messaging user interface 1200 of the messagingapplication while the live communication session has is being initiated.Messaging user interface 1200 includes message 1212 in message region1202 and visual indicator 1214 at the top of display 602 indicating thatthe live communication session is active.

In some embodiments, in response to receiving user input 1250A, device600 launches the live communication application. As illustrated in FIG.12C, after the live communication session begins, device 600 displayslive communication user interface 1216 of the live communicationapplication. Live communication user interface 1216 corresponds to thelive communication session started with the group The Dream Team.

As illustrated in FIG. 12C, live communication user interface 1216includes primary representations 1210A-1210D (collectively 1210) of fourcorresponding participants (e.g., the first four participants in a listof participants of the group The Dream Team) and user representation1218 of the user of device 600. Live communication user interface 1216also includes overflow region 1220 with overflow representations 1222corresponding to participants of the live communication session (e.g.,each of the members of The Dream Team group). Primary representation1210B includes a live video from participant Allison, indicating thatAllison has joined the live communication session with video data.

Primary representations 1201A, 1210C, and 1210D include placeholdersindicating that the corresponding participants have not joined (e.g.,are not connected to) the live communication session. The placeholdersinclude a textual representation of the participant's name (e.g., firstname or first and last initial). Optionally, the placeholders include astatus indicator, indicating that the participant is waiting to connectto the live communication session, and a reminder affordance (e.g.,“RING” 808A) for sending a reminder to the corresponding participantthat the live communication session is available. In some embodiments,reminder affordances are not initially included on the placeholders andare displayed in accordance with a determination that the correspondingparticipant has not joined the live communication session after apredetermined period of time.

In some embodiments, device 600 detects an input corresponding toselection of a reminder affordance (e.g., a tap on a reminderaffordance), and in response, sends instructions to cause a newnotification (e.g., call, ring, etc.) indicating that the livecommunication session is available at a device associated with thecorresponding participant, where the new notification is different thana previous notification sent to the participant. In some embodiments,the new notification is more intrusive than the previous notification(e.g., an audio ring, haptic output, or full screen notification, ascompared to a banner notification without audio or haptic output). Insome embodiments, reminder affordances are displayed in overflowrepresentations 1222 (e.g., in accordance with the same criteria usedfor displaying reminder affordances in the primary representations). Insome embodiments, selection of a reminder affordance on an overflowrepresentation results in the same response (e.g., a new, more intrusivenotification to the corresponding participant) as described above withrespect to the reminder affordances displayed on the primaryrepresentations.

In some embodiments, in accordance with a determination that one of theparticipants represented by a placeholder has joined the livecommunication session, device 600 modifies the representation associatedwith the user (e.g., replaces the placeholder with live video; see,e.g., FIGS. 8C-8D). In some embodiments, modifying the representationincludes an animation. In some embodiments, device 600 receives datafrom a server or other device to determine whether a participant hasjoined the session. In some embodiments, device 600 determines that aparticipant has joined the session based on receiving live media dataassociated with the participant.

In some embodiments, in accordance with a determination that one of theparticipants represented by a placeholder has not joined the livecommunication session for a predetermined amount of time, device 600changes the layout of the representations displayed in livecommunication session user interface 1216 (e.g., device 600 ceases todisplay the representation corresponding to the participant that has notjoined the live communication session for a predetermined amount of timeand displays a layout with fewer primary representations or replaces therepresentation of the participant with a representation of anotherparticipant; see, e.g., FIGS. 8E-8F). In some embodiments, device 600receives data from a server or other device to determine whether aparticipant has not joined the live communication session for apredetermined amount of time. In some embodiments, device 600 determineswhether a participant has not joined the live communication session fora predetermined amount of time based on not receiving live media dataassociated with the participant.

FIG. 12D illustrates live communication user interface 1216 at a latertime during the live communication session with the group The DreamTeam. As illustrated in FIG. 12D, live communication user interface 1216includes call control menu 1224, which is analogous to call control menu630 described above. As described above, in some embodiments, callcontrol menu 1224 is displayed in response to a user input (e.g., a tapor swipe up on display 602). As illustrated in FIG. 12D, device 600receives (e.g., detects) user input 1250B (e.g., a swipe up)corresponding to a request to expand call control menu 1224. Asillustrated in FIG. 12E, in response to user input 1250B, device 600expands call control menu 1224. Expanded call control menu 1224 isanalogous to expanded call control menu 630 described with respect toFIG. 6I-6K. As illustrated in FIG. 12E, call control menu 1224 includesreminder affordance 1226 associated with the participant Hoan. In someembodiments, device 600 detects an input corresponding to selection ofreminder affordance 1226 (e.g., a tap on reminder affordance 1226), andin response, sends instructions to cause a new notification (e.g., call,ring, etc.) indicating that the live communication session is availableat a device associated with the participant Hoan, where the newnotification is different than a previous notification sent to theparticipant Hoan. In some embodiments, the new notification is moreintrusive than the previous notification (e.g., an audio ring, hapticoutput, or full screen notification, as compared to a bannernotification without audio or haptic output).

FIG. 12F again illustrates live communication user interface 1216 priorto starting the live communication session with the message group TheDream Team. As illustrated in FIG. 12F, device 600 receives (e.g.,detects) user input 1250C (e.g., a tap) corresponding to selection ofgroup contact affordance 1208. As illustrated in FIG. 12G, in responseto receiving user input 1250C, device 600 displays group contact userinterface 1228, which shows additional information related to The DreamTeam group including live communication session affordance 1230 and list1232 of the participants in the group. Group contact user interface 1228also includes add contact affordance 1234. In some embodiments, inresponse to detecting an input (e.g., a tap) corresponding to theselection of add contact affordance 1234, device 600 initiating aprocess for adding a new contact to the live communication session(e.g., by displaying a user interface with a text entry field forentering the names of participants using a keyboard and/or an addparticipant affordance for selecting participants from a menu such as acontacts list (e.g., as described below with reference to FIG. 14F)).

As illustrated in FIG. 12H, device 600 receives (e.g., detects) userinput 1250D (e.g., a tap) corresponding to selection of livecommunication session affordance 1230. In response to receiving userinput 1250D, device 600 launches the live communication application andstarts the live communication session with the participants in the groupThe Dream Team (e.g., as illustrated in FIG. 12C).

A user can also select a representation of a participant in the group ongroup contact user interface 1228 to access additional information andoptions related to the corresponding participant. As illustrated in FIG.12I, device 600 receives (e.g., detects) user input 1250E (e.g., a tap)corresponding to selection of list item 1232B for participant Allison.In response to user input 1250E, device 600 displays individual contactuser interface 1236 with additional information associated with theselected participant and options 1238A-1238E for communicating with theparticipant individually via various communication modes. As illustratedin FIG. 12J, individual contact user interface 1236 includes, interalia, affordances 1238A, 1238B, and 1238C for initiating, respectively,an individual message conversation, an individual telephone call, and anindividual live communication session (e.g., a video communication) withparticipant Allison.

In some embodiments, device 600 determines whether a characteristicintensity of user input 1250E exceeds a threshold intensity. Asillustrated in FIG. 12K, in accordance with a determination that thecharacteristic intensity of user input 1250E exceeds the thresholdintensity, device 600 displays menu 1240 on top of group contact userinterface 1228 with options 1242A-1242E corresponding to variouscommunication modes for individual communication with the participantAllison.

FIGS. 12L-12N illustrate exemplary 4-on-1, 3-on-1, and 2-on-1 layouts,respectively. In some embodiments, in accordance with a livecommunication session including five participants (including the user ofdevice 600), device 600 displays one of the initial layouts illustratedin FIG. 12L. In some embodiments, in accordance with a livecommunication session including four participants (including the user ofdevice 600), device 600 displays one of the initial layouts illustratedin FIG. 12M. In some embodiments, in accordance with a livecommunication session including three participants (including the userof device 600), device 600 displays one of the initial layoutsillustrated in FIG. 12N. In some embodiments, device 600 displaysrepresentations of participants in one of the layouts illustrated inFIGS. 12L-12N as an initial layout when initiating a live communicationsession (e.g., instead of the layout illustrated in FIG. 12C) based onthe number of invited and/or connected participants. For example, insome embodiments, in accordance with a live communication sessionincluding a total of four participants, device 600 displaysrepresentations of the participants according to one of the layoutsillustrated in FIG. 12M.

FIG. 13A-13D is a flow diagram illustrating a method in accordance withsome embodiments. Method 1300 is performed at a device (e.g., 100, 300,500, or 600) with a display. Some operations in method 1300 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 1300 provides an intuitive way for initiatingand interacting with a live communication session between multipleparticipants. The method reduces the cognitive burden on a user byproviding the intelligent initiation of and interaction with a livecommunication session between multiple participants, thereby creating amore efficient human-machine interface. For battery-operated computingdevices, enabling a user to interact with and initiate a livecommunication session faster and more efficiently conserves power andincreases the time between battery charges.

At block 1302, the device displays a messaging user interface (e.g.,1200) of a messaging application. The messaging user interface includesa message region (e.g., 1202) including a plurality of messages betweenparticipants in a message conversation that includes three or moreparticipants. The messaging user interface includes an affordance (e.g.,1206) separate from the message region (e.g., 1202) for starting a livevideo communication session in a live video communication application(e.g., a message details affordance or an affordance with a graphicalindication of the live video communication application). In someembodiments, the affordance is a call-all affordance (e.g., 1206) thatwill immediately start a live communication session with all of theparticipants in the messaging application. In some embodiments, theaffordance is a group contact affordance (e.g., 1208) for displaying agroup contact user interface to invite participants to a livecommunication session.

At block 1340, the device detects an input (e.g., 1250A) correspondingto selection of the affordance (e.g., 1206). At block 1306, the device,in response to detecting the input corresponding to selection of theaffordance, initiates a process for starting the live videocommunication session in the live video communication application, thelive video communication session including the participants of themessage conversation. In some embodiments, the live video communicationapplication is launched (e.g., 1216), and the new live videocommunication session is started directly in response to selection ofthe affordance. In some embodiments, a menu of the messaging application(e.g., 1228) is displayed in response to selection of the affordance anda menu option (e.g., 1230) is selected to launch the live videocommunication application and start the new live video communicationsession. Automatically initiating a process for starting the live videocommunication session in the live video communication application whenprescribed conditions are met (e.g., by detecting an input) allows auser to start a live communication session. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, providingan additional control (e.g., an affordance) to initiate the process forstarting the live communication provides additional control options(e.g., an affordance) without cluttering the UI with additionaldisplayed controls (e.g., a group chat UI) enhances the operability ofthe device and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1308, the device transitions (e.g., immediatelytransitioning in response to detecting the user input) from themessaging user interface (e.g., 1200) to a user interface (e.g., 1216)of the live communication application. Optionally, at block 1310, thedevice starts the live communication session. Optionally, at block 1312,the device displays a group contact user interface (e.g., 1228), thegroup contact user interface (e.g., an interface displayed whilecontinuing to display at least a portion of the messaging user interfaceor a discrete user interface that replaces the messaging user interface)includes a group contact affordance (e.g., 1230) for starting the livecommunication session. Automatically transitioning from the messaginguser interface and starting the live video communication session in thelive video communication application when prescribed conditions are met(e.g., by detecting an input) allows a user to start a livecommunication session immediately. Performing an optimized operationwhen a set of conditions has been met without requiring further userinput enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In addition, transitioning from the messaging userinterface and starting the live video communication session in the livevideo communication application when prescribed conditions are metreduces the number of inputs needed to perform an operation. Reducingthe number of inputs needed to perform an operation enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some embodiments, the group contact user interface further includes:a plurality of participant affordances (e.g., 1232) including a firstparticipant affordance (e.g., 1232B) that is associated with a firstparticipant in the participants of the message conversation (e.g., listof individual participants). In some embodiments, the group contact userinterface further includes a new contact affordance (e.g., 1234).Optionally, at block 1314, the device starts the live communicationsession in the live communication application. Displaying a groupcontact user interface provides additional control options (e.g., anaffordance) without cluttering the UI with additional displayed controlswhen the additional controls are needed enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1316, the device displays a live communicationsession user interface (e.g., 1216) (e.g., an interface for livecommunication placeholders) that includes (e.g., initially includes) aplaceholder (e.g., 1210A of FIG. 12C) (e.g., a graphical placeholder(e.g., an image or virtual avatar) a textual placeholder (e.g., a nameor initials) associated with a first participant (e.g., a participantother than the user of the electronic device. A participant that has yetto join the communication session and that has been sent an invitationto join the communication session) of the live communication sessionparticipants. In some embodiments, the live communication session userinterface further includes a second placeholder associated with a secondparticipant. In some embodiments, the first placeholder includes a firstreminder affordance (e.g., a ring button) for the first participant ofthe live communication session participants (e.g., FIG. 12C). In someembodiments, the live communication session user interface furtherincludes a second placeholder associated with a second participant(e.g., 1210C). In some embodiments, the first placeholder and the secondplaceholder are arranged according to a first layout. Displaying a livecommunication session user interface when the live communication isstarted provides the user with feedback about the current state of thelive communication session and provides visual feedback to the userindicating that operations associated with the live communicationinterface may be performed. Providing improved visual feedback to theuser enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at block 1318, in accordance with a determination that thefirst participant has joined the live communication session, the devicereplaces display of the first placeholder (e.g., 1210A of FIG. 12C) witha representation (e.g., 1210A of FIG. 12D) of the first participant(e.g., a placeholder can have participants name). In some embodiments,an animation occurs (e.g., FIGS. 8T-8W). In some embodiments, the devicereceives data from server or another device to make this determination.In some embodiments, the device makes this determined because it hasn'tdisplayed any representations of the user. Modifying the display of thefirst placeholder with a representation of the participant allows theuser to quickly recognize which users have joined the call, provides theuser with feedback about the current state of a live communicationsession, and provides visual feedback to the user indicating thatcertain operations associated with the notification and/or the livecommunication session will be performed. Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at block 1320, the device, in accordance with adetermination that the first participant has not joined the livecommunication session for a predetermined amount of time (e.g., notjoined the live communication after a predetermined period of time), thedevice ceases (optionally at block 1322) display the first placeholderand, optionally at block 1324, the device changes the arrangement of thesecond placeholder to a second layout, different than the first layout(e.g., display a layout with the first participant not included in thelayout). In some embodiments, the device can receive data from server oranother device to make this determination. In some embodiments, thedevice can make this determined because it hasn't displayed anyrepresentations of the user. Ceasing to display the placeholder andchanging the arrangement of the display when prescribed conditions aremet allows users to quickly recognize when other users are joining ornot joining the call while maximizing the visibility of displayedparticipants and/or placeholders. Performing an optimized operation whena set of conditions has been met without requiring further user inputenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In addition, ceasing to display the placeholder andchanging the arrangement of the display provides the user with feedbackabout the current state of a live communication session, and providesvisual feedback to the user indicating that certain operationsassociated with the notification and/or the live communication sessionwill be performed. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 1326 and 1328, the device receives an input (e.g.,850B) corresponding to selection of the first reminder affordance (e.g.,808). In response to receiving the input corresponding to selection ofthe first reminder affordance, the device sends a new notification(e.g., call, ring, etc.) to the first participant that is different thana previous notification sent to the first participant (e.g., the secondnotification is more intrusive than the first notification). Sending anotification to the first participant in response to receiving an inputcorresponding to a selection of a reminder affordance that is differentfrom a previous notification sent to the participant reduces the numberof inputs needed to perform an operation. Reducing the number of inputsneeded to perform an operation (e.g., the number of options needed forthe user to call the participant) enhances the operability of the deviceand makes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, performingan operation automatically, such as sending a notification that isdifferent from a first notification, without further user input)enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 1330 and 1332, the device detects an input (e.g.,1250C) corresponding to selection of the group contact affordance (e.g.,1208) for starting the live communication session. In response todetecting the input corresponding to selection of the group contactaffordance for starting the live communication session, the devicestarts the live communication session in the live communicationapplication. (e.g., transitioning to the user interface of the livecommunication application). In some embodiments, the group contact userinterface includes a listing (e.g., 1232) of the participants for thelive communication session. In some embodiments, the listing of theparticipants is provided without providing options and/or affordancesfor individually joining or starting a video or audio conference withthe given participant. In some embodiments, the messaging user interfaceincludes, two affordances for starting a live video communicationsession in a live video communication application: a first affordancethat, when selected, results in display of the group contact userinterface, and a second affordance that, when selected, results intransition to a user interface of the live communication application andin starting (e.g., immediately starting) the live communicationsession). Starting the live communication session in the livecommunication application in response to the selection of a groupcontract affordance provides additional control options (e.g., anaffordance) without cluttering the UI with additional displayed controlswhen the additional controls are needed enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, startingthe live communication session in the live communication application inresponse to the selection of a group contract affordance reduces thenumber of inputs needed to perform an operation. Reducing the number ofinputs needed to perform an operation enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 1334, 1336, 1338, and 1340, the device, afterstarting the communication: in response to a determination that thefirst participant has not joined the live communication session for apredetermined period of time, displays a reminder option that isassociated with the first participant In some embodiments, the reminderoption is displayed on or near the first participant affordance) in thegroup contact user interface. Displaying a reminder option that isassociated with a participant when prescribed conditions are met allowsthe user to quickly recognize users that have not joined the call andcall those particular users. Performing an optimized operation when aset of conditions has been met without requiring further user inputenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. While displaying the reminder option that is associatedwith the first participant, the device receives an input correspondingto selection of the reminder option. At, in response to receiving theinput corresponding to selection of the reminder option, the devicesends a notification (e.g., a notification that causes a device of thefirst participant to output (e.g., an audible, visual, haptic) anotification) indicating that the live communication session isavailable to the first participant. Sending a notification to the firstparticipant in response to receiving an input corresponding to aselection of a reminder option that is associated with the participantreduces the number of inputs needed to perform an operation. Reducingthe number of inputs needed to perform an operation (e.g., the number ofoptions needed for the user to call the participant) enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

Optionally, at blocks 1342 and 1344, the device detects an input (e.g.,1250E) corresponding to selection of the first participant affordance(e.g., 1232B) that is associated with the first participant of themessage conversation. In some embodiments, these participants areparticipants of the live communication session (e.g., participants ofthe live communication sessions can include one or more different usersthan those from the message conversation and can also include none ofthe participants from the message conversation). In response todetecting the input corresponding to selection of the first participantaffordance that is associated with the first participant of the messageconversation, the device displays an individual participant userinterface (e.g., 1236) that includes a characteristic of the firstparticipant (e.g., the individual participant user interface has detailsconcerning the individual user (e.g., user identifiers, such as names,initials, etc.; user contact information; modes of communication tocontact the user)). Displaying a individual participant user interfacein response to detecting an input (e.g., a participant affordance foreach individual participant on the group contact UI) without clutteringthe UI (e.g., group contact UI) with additional displayed controls whenthe additional controls are needed enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 1346 and 1348, the device detects an input (e.g.,an input having a characteristic intensity that exceeds an intensitythreshold) corresponding to selection of the option for selecting themode of communication (e.g., 1238A-1238C). In response to detecting theinput corresponding to selection of the mode of communication option,the device initiates a process to select the mode of commutation for thefirst participant. In some embodiments, initiating the process includesdisplaying a plurality of communication option affordances (e.g., a mailaffordance, a call affordance, a messaging affordance)).

Optionally, at blocks 1350 and 1352, the device detects an inputcorresponding to selection of the new contact affordance (e.g., 1234).In response to detecting the input corresponding to selection of the newcontact affordance, the device initiates a process for adding a newcontact to the live communication session (e.g., displaying a userinterface for selecting a new contact to add). Initiating a process foradding a new contact to the live communication session allows the userto quickly recognize which users are added to the call, provides theuser with feedback about the current state of a live communicationsession, and provides visual feedback to the user indicating thatcertain operations associated with the notification and/or the livecommunication session will be performed. Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Note that details of the processes described above with respect tomethod 1300 (e.g., FIGS. 13A-13D) are also applicable in an analogousmanner to the methods described below/above. For example, method 1300optionally includes one or more of the characteristics of the variousmethods described with reference to method 700, 900, 1100, or 1500.

FIGS. 14A-14I illustrate exemplary user interfaces for, inter alia,initiating a live communication session from a live communicationapplication, in accordance with some embodiments. The user interfaces inthese figures are used to illustrate the processes described below,including the processes in FIGS. 15A-15C.

FIG. 14A illustrates device 600 (as described above) displaying userinterface 1400 (e.g., a home screen or springboard) with affordances forlaunching applications. As illustrated in FIG. 14A, device 600 receives(e.g., detects) user input 1450A (e.g., a tap) corresponding toselection of live communication application affordance 1402.

As illustrated in FIG. 14B, in response to receiving user input 1450A,device 600 launches the live video communication application anddisplays live video communication user interface 1404 of the live videocommunication application. The live video communication user interfaceincludes list 1408 of participant affordances 1408A-1408K correspondingto respective participants or participant groups. In some embodiments, aparticipant affordance is included in list 1408 based on a previous(e.g., recent) live communication session with the respectiveparticipant or participant group (e.g., two or more participants otherthan the user of device 600) in the live communication application. Insome embodiments, previous live communication sessions includesuccessfully connected and attempted communications, either by the userof device 600 or another participant. Live video communication userinterface 1404 includes new session affordance 1406 for selectingparticipants for a new live communication session.

In some embodiments, a participant affordance is included in list 1408based on a participant group associated with another application (e.g.,a messages application (affordance 1408A; e.g., a group chat participantlist), phone application (affordance 1408C; e.g., participants of aconference call), calendar application (affordance 1408B; e.g.,attendees of a calendar event), or email application (affordance 1408D;e.g., a recipient list of an email)). For example, participantaffordance 1408A corresponds to a group of participants in a previous orexisting group message conversation (e.g., text or instant message) andis displayed in list 1408 as a suggestion (e.g., a user did not have tomanually create the participant group). In some embodiments, aparticipant affordance corresponds to a group of participants in atelephonic conference call associated with a phone application.Participant affordances 1408A-1408K include an indication of theparticipant or participant group, a status indicator, a detailsaffordance, and, optionally, an associated date. As illustrated in FIG.14B, the status indicator identifies a source of a participant or group(e.g., “From Messages”, “From Phone”, “From Calendar”, “From Mail”) or acommunication status of an associated communication (e.g., video, audio,unavailable, etc.). In some embodiments, the source of a participant orgroup is indicated by a graphical or textual indication, a colorassociated with a certain application, or a logo related to theapplication (e.g., a phone icon, video camera icon, or message bubbleicon).

In some embodiments, device 600 visually distinguishes a participantaffordance associated with a group for which there is an active livecommunication session. In FIG. 14B, participant affordance 1408E isassociated with a group for which a live communication session iscurrently active. The active live communication session is indicated bythe bold text for the participant names, video camera icon 1408E-1, and“TAP TO JOIN” message in participant affordance 1408E, as compared tothe non-bold text and contact details icon (e.g., 1408A-1) displayed inother participant affordances.

In some embodiments, the appearance of a participant affordanceassociated with a previous live communication session is based onwhether or not the user of the device joined the corresponding livecommunication session. In FIG. 14B, participant affordance 1408H isassociated with a previous live communication session that the user ofdevice 600 did not join. As a result, participant affordance 1408Hincludes italicized text, as comparted to the non-italicized text ofother participant affordances associated with previous livecommunication session that the user of device 600 joined (e.g., 1408F).

As illustrated in FIG. 14C, device 600 receives (e.g., detects) userinput 1450B (e.g., a tap) corresponding to selection of participantaffordance 1408A for participant group The Dream Team described above,which includes 15 participants.

In response to receiving user input 1450B, device 600 initiates aprocess of starting a new live video communication session with theparticipants corresponding to selected participant affordance 1408A (TheDream Team). In some embodiments, the new live video communicationsession is started directly in response to selection of participantaffordance 1408A. For example, as illustrated in FIG. 14D, in responseto receiving user input 1450B, device 600 displays live communicationuser interface 1410 and initiates the live communication session withthe group The Dream Team without further user input. Live communicationuser interface 1410 includes a representation of the user of device 600(e.g., a video from a camera on the front of device 600) and callcontrol menu 1412, analogous to call control menu 630 discussed.

In some embodiments, a menu of the live video communication applicationis displayed in response to selection of participant affordance 1408Aand a menu option is selected to start the new live video communicationsession. For example, in some embodiments, device 600 displays livecommunication user interface 1410 (as shown in FIG. 14D) withoutinitiating the live communication session. Instead, while displayinglive communication user interface 1410, device 600 receives (e.g.,detects) a user input (e.g., a tap) corresponding to selection of callaffordance 1414 on call control menu 1412, and initiates a livecommunication session with the group The Dream Team in response toselection of live communication initiation affordance 1414.

FIG. 14E also illustrates live communication user interface 1404.Instead of receiving selection of a participant affordance in list 1408(as illustrated in FIG. 14C), device 600 receives (e.g., detects) userinput 1450C (e.g., a tap) corresponding to selection of new sessionaffordance 1406. In response to user input 1450C, and in accordance withuser input 1450C corresponding to selection of new session affordance1406, device 600 initiates a process for selecting participants andstarting a new live video communication session between the selectedparticipants and the user of device 600. As illustrated in FIG. 14F,live communication user interface 1404 is replaced with user interface1416, which includes text entry field 1418 for entering the names ofparticipants using keyboard 1420, add participant affordance 1422 forselecting participants from a menu (e.g., a contacts list), audio-onlyaffordance 1424, and video affordance 1426.

In FIG. 14F, in response to entry of text “JOE” into text entry field1418, device 600 displays two communication options 1417A and 1417B forcontact “JOE SMITH.” In some embodiments, device 600 visuallydistinguishes communication options that are associated with a devicecapable of joining a live communication session. In FIG. 14F, thecommunication mode (e.g., email) represented by communication option1417B is associated with a device capable of joining a livecommunication session, as indicated by bold text. In contrast, thecommunication mode (e.g., home phone) represented by communicationoption 1417A is displayed in normal (non-bold) text to indicate that thecommunication mode is not associated with a device capable of joining alive communication session.

As illustrated in FIG. 14G, participants named Joe, Ashley, and Sam havebeen selected. Device 600 receives (e.g., detects) user input 1450D(e.g., a tap) corresponding to selection of video affordance 1426.

In response to receiving user input 1450D, device 600 starts a new livecommunication session with the selected participants and provides a livemedia stream including video and audio data (as illustrated in FIG.14H). In some embodiments, in response to receiving user inputcorresponding to selection of audio only affordance 1424, device 600starts a new live communication session with the selected participantsand provides a live media stream including audio data, without videodata.

In some embodiments, after the new live communication session isstarted, device 600 displays a representation of the new livecommunication session in an application other than the livecommunication application. After starting the live communication sessionwith participants Joe, Ashley, and Sam, the user of device 600 navigatesto a messaging application. As illustrated in FIG. 14I, after startingthe live communication session, device 600 displays messaging userinterface 1428 of the messaging application. Messaging user interface1428 displays a group message conversation including the participants ofthe live communication session, and message 1430 indicating that thelive communication session has been started is displayed in messageregion 1432. In some embodiments, after starting the live communicationsession, device 600 displays a group including the participants of thelive communication session in a list of message conversations in themessaging application. In some embodiments, device 600 displaysmessaging user interface 1428 (described above) in response to selectionof the group in the list of message conversations.

FIGS. 15A-15C are a flow diagram illustrating a method in accordancewith some embodiments. Method 1500 is performed at a device (e.g., 100,300, 500, or 600) with a display. Some operations in method 1500 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 1500 provides an intuitive way for initiatingand interacting with a live communication session between multipleparticipants. The method reduces the cognitive burden on a user forproviding the intelligent initiation of and interaction with a livecommunication session between multiple participants, thereby creating amore efficient human-machine interface. For battery-operated computingdevices, enabling a user to interact with and initiate a livecommunication session faster and more efficiently conserves power andincreases the time between battery charges.

At block 1502, the device displays a live video communication userinterface (e.g., 1404) of a live video communication application. Atblock 1504, the live video communication user interface includes aplurality of participant affordances (e.g., 1408A-1408K) correspondingto respective one or more participants, the plurality of participantaffordances including a first participant affordance (e.g., 1408A)corresponding to a group of three or more participants. At block 1506,the live video communication user interface includes a new sessionaffordance (e.g., 1406) for starting a new live video communicationsession. In some embodiments, a participant affordance corresponds to aprevious (e.g., recent) video communication (e.g., successfullyconnected or attempted (either by the participant using the device or bya participant associated with an external device)) and includes anindication of the participant(s) of the video communication (e.g.,1408F). In some embodiments, a participant affordance represents a groupassociated with another communication application (e.g., a messagingapplication or a phone application (e.g., a recent conference call)). Insome embodiments, the live communication user interface includes a listof recent live communication sessions (e.g., list of recent video chatscan be individual and/or group chats). Displaying a live communicationuser interface that includes a list of recent live communicationsessions allows user to quickly access and realize the recent livecommunication sessions that the user or device involved in (e.g.,invited to, joined, communication in, etc.), provides the user withfeedback about the current state of a live communication session, andprovides visual feedback to the user indicating that certain operationsassociated with the notification and/or the live communication sessionwill be performed. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, the plurality of participant affordances includes asecond participant affordance corresponding to a suggested group ofparticipants (e.g., 1408B) (e.g., a suggested group of participants thatwas not manually selected/created by a user) determined based on data(e.g., usage data) from a second application (e.g., a phone application,a messaging application) different than the live video communicationapplication.

In some embodiments, the suggested group of participants corresponds toparticipants in a group conversation (e.g., a group messaging session,group conference call) conducted in the second application (e.g., 1408A)or a conference call conducted in the second application (e.g., 1408C).Displaying a live communication user interface that includes anaffordance corresponding to a suggested group of participants allowsusers to quickly access and start a live communication with a suggestedgroup and reduces the number of inputs needed to perform an operationassociated with setting up a live communication session that includeseach member of the selected group. Reducing the number of inputs neededto perform an operation enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, displayingan affordance corresponding to a suggested group provides additionalcontrol operation without cluttering the UI with addition displayedcontrols. Providing additional control operation without cluttering theUI with addition displayed controls enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the second participant affordance includes anindication (e.g., a graphical or textual indication (e.g., “suggestedfrom Messages”) a color related to a certain application, or a logorelated to the application) of the second application (e.g., the sourceapplication for the suggestion). Displaying a live communication userinterface that further includes an affordance corresponding to a suggestgroup of participants that corresponds to participants in a groupconversation in a different application than the live communicationapplication allows users to quickly access and start a livecommunication with a suggested group and reduces the number of inputsneeded to perform an operation associated with setting up a livecommunication session that includes each member of the selected group(e.g., such as navigating the different application). Reducing thenumber of inputs needed to perform an operation enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some embodiments, theplurality of participant affordances includes a third participantaffordance corresponding to a second suggested group of participants(e.g., a suggested group of participants that was not manuallyselected/created by a user) determined based on data (e.g., usage data)from a third application (e.g., a phone application, a messagingapplication, an email application, a calendar application) differentthan the live video communication application and different from thesecond application (e.g., a group from a recent calendar event or agroup from a recent mail event). Displaying a second participantaffordance that includes an indication of an application that thesuggested group is from provides the user with feedback about thecurrent state of a live communication session and provides visualfeedback to the user indicating that certain operations associated withthe second application. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice), which additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In some embodiments, the live communication userinterface further includes a participant selection affordance (e.g.,“+”).

At blocks 1508, the device detects an input (e.g., 1450B) thatcorresponds to a request to initiate a process of starting a new livevideo communication session. At blocks 1510, 1512, and 1514, the device,in response to detecting the input: in accordance with the inputcorresponding to selection of the first participant affordance (e.g.,1408A), initiates a process of starting new live video communicationsession that includes the group of three or more participants. In someembodiments, the new live video communication session is starteddirectly in response to selection of the first participant affordance(e.g., FIG. 14D). In some embodiments, a menu (e.g., 1416) of the livevideo communication application is displayed in response to selection ofthe first participant affordance, and a menu option is selected to startthe new live video communication session. In response to detecting theinput, the device, in accordance with the input corresponding toselection of the new session affordance, initiates a process forselecting two or more participants and starts a new live videocommunication session that includes a participant associated with thedevice and the two or more selected participants (e.g., FIG. 14F). Insome embodiments, the user interface is replaced with another userinterface that includes a first affordance (e.g., 1422) for selectingparticipants and a second affordance (e.g., 1424 and 1426) for startingthe live video communication session (e.g., starting the live videocommunication session directly in response to selection of the secondaffordance). In some embodiments, participants are selected by enteringcontact information into a text field (e.g., 1418) or selecting acontact from a list of contacts. In some embodiments, the new live videocommunication session is started directly in response to selection ofthe affordance (e.g., 1408A). Initiating a process for starting a newlive communication when prescribed conditions are met allows the deviceto provide the user with different live video communication sessionbased on the number of selected participants. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In addition, initiatingdifferent processes for starting a new live communication whenprescribed conditions are met reduces the number of inputs needed toperform an operation. Reducing the number of inputs needed to perform anoperation enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

Optionally, at blocks 1516, 1518, 1520, 1522, 1524, and 1526, the devicereceives a second input corresponding to selection of the participantselection affordance. In response to receiving the second user inputcorresponding to selection of the participant selection affordance, thedevice displays a group selection user interface (e.g., 1416) thatincludes: a text entry field (e.g., 1418) for identifying one or moreparticipants (e.g., to: field); a keyboard (e.g., 1420); an audio-onlyaffordance (e.g., 1424); and a video affordance (e.g., 1426). Whiledisplaying the group selection user interface, the device receives athird user input selection. In response to receiving the third userinput, the device, in accordance with the third user input correspondingto selection of the audio-only affordance, starts the new livecommunication session using audio transmitted from the electronic device(e.g., audio recorded by one or more microphones of the device), withouttransmitting video data from the electronic device. Starting the newlive communication session using audio transmitted from the electronicdevice when prescribed conditions are met (e.g., in response to anaudio-only affordance being selected) allows a user to join livecommunication sessions using audio without the user having to select ormodify additional input and outputs of the device (e.g., turning offcamera and/or turning on microphone). Performing an optimized operationwhen a set of conditions has been met without requiring further userinput enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In some embodiments, the system displays an indicatorthat describes that the user has joined the live communication usingaudio only. In some embodiments, the device prompts user display aphrase such as “do you want to connect with video” in order to determinewhether the user wants to use audio only or video and audio. This cangive the user comfort in that video is not being shared). In response toreceiving the third user input, in accordance with the third user inputcorresponding to selection of the video affordance, the device startsthe new live communication session using audio and video transmittedfrom the electronic device. Starting the live communication sessionusing audio and video transmitted from the electronic device whenprescribed conditions are met (e.g., in response to a video affordanceoption being selected) allows a user to join live communication sessionsusing audio and video without the user having to select or modifyadditional input and outputs of the device (e.g., turning on cameraand/or turning on microphone). Performing an optimized operation when aset of conditions has been met without requiring further user inputenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Optionally, at blocks 1528 and 1530, the device starts the new livecommunication session. After starting the new live communicationsession, the device displays a representation of the new livecommunication session in a fourth application (e.g., messages or phoneapplications) that is different from the live communication session. Insome embodiments, live communication participants are displayed as groupparticipants in a group message in a list of message conversation in themessaging application. In some embodiments, the group message in thelist of message conversations includes an indication of whether there isan active live communication session including the group participants(e.g., 1408E). In some embodiments, live communication participants aredisplayed as group participants in a group call in a list of recentcalls in the phone application or call application. In some embodiments,live communication participants are displayed as invitees to a calendarevent in a calendar application. Displaying a representation of the livecommunication session in an application that is different than the livecommunication session provides the user with feedback about the currentstate of a live communication session in multiple applications, providesvisual feedback to the user indicating that certain operationsassociated with the notification and/or the live communication sessionwill be performed, if the user merely, in some embodiments, activatesthe representation at one time (e.g., starting the another new livecommunication session in the live communication application from anotherapplication with the same participants that were in the previous livecommunication session at a later time). Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the plurality of participant affordances (e.g.,1408) includes a fourth participant affordance (e.g., 1408A, 1408E)corresponding to a group of three or more participants. In someembodiments, in accordance with a determination that the fourthparticipant affordance corresponds to an active live communicationsession (e.g., 1408E), the fourth participant affordance is displayedwith a first visual appearance (e.g., the first visual appearance isfont color or font style of the participant affordance or an icon orglyph displayed at or near the participant affordance); and inaccordance with a determination that the fourth participant affordancecorresponds to a non-active live communication session (e.g., 1408A),the fourth participant affordance is displayed with a second visualappearance different than the first visual appearance (e.g., the secondvisual appearance is a font color or font style that is different than afont color or font style of the first visual appearance, or the secondvisual appearance does not include an icon or glyph displayed near therepresentation of the potential participant or includes a different iconor glyph than the first visual appearance). In some embodiments, theappearance of a participant affordance associated with a previous livecommunication session is based on whether or not the user of the devicejoined the corresponding live communication session (e.g., theparticipant affordance for a missed live communication session (e.g.,1408H) includes a font color, font style, icon, glyph, etc. that isdifferent than or not included in a participant affordance (e.g., 1408F)for a live communication session that the user of device 600 joined; aparticipant affordance includes text of one color if the user of thedevice joined the session and includes text of a different color if theuser of the device did not join the session before it ended). In someembodiments, the appearance of a participant affordance associated withan active live communication session changes when the session is nolonger active (e.g., from bold text and a video camera icon to non-boldtext without a video camera icon).

In some embodiments, initiating the process for selecting two or moreparticipants includes displaying a representation (e.g., 1417A, 1417B)of a potential participant (e.g., a contact in a contact list). In someembodiments, in accordance with a determination that the potentialparticipant is associated with a device capable of joining a livecommunication session, the representation of the potential participantis displayed with a first visual appearance (e.g., the first visualappearance is font color or font style of the representation of thepotential participant or an icon or glyph displayed near therepresentation of the potential participant); and in accordance with adetermination that the participant is not associated with a devicecapable of joining a live communication session, the representation ofthe potential participant is displayed with a second visual appearancedifferent than the first visual appearance (e.g., the second visualappearance is a font color or font style that is different than a fontcolor or font style of the first visual appearance, or the second visualappearance does not include an icon or glyph displayed near therepresentation of the potential participant or includes a different iconor glyph than the first visual appearance).

Note that details of the processes described above with respect tomethod 1500 (e.g., FIG. 15A-15C) are also applicable in an analogousmanner to the methods described above. For example, method 1500optionally includes one or more of the characteristics of the variousmethods described above with reference to method 700, 900, 1100, or1300.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the techniques and their practical applications. Othersskilled in the art are thereby enabled to best utilize the techniquesand various embodiments with various modifications as are suited to theparticular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to improve themultimedia communication. The present disclosure contemplates that insome instances, this gathered data may include personal information datathat uniquely identifies or can be used to contact or locate a specificperson. Such personal information data can include demographic data,location-based data, telephone numbers, email addresses, twitter ID's,home addresses, data or records relating to a user's health or level offitness (e.g., vital signs measurements, medication information,exercise information), date of birth, or any other identifying orpersonal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used tofacilitate multimedia communications between users. Accordingly, use ofsuch personal information data enables users to improve suchcommunications. Further, other uses for personal information data thatbenefit the user are also contemplated by the present disclosure. Forinstance, health and fitness data may be used to provide insights into auser's general wellness, or may be used as positive feedback toindividuals using technology to pursue wellness goals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof multimedia communications, the present technology can be configuredto allow users to select to “opt in” or “opt out” of participation inthe collection of personal information data during registration forservices or anytime thereafter. In another example, users can select notto provide location data and/or availability. In yet another example,users can select to limit the length of time records of communicationsessions are maintained or entirely prohibit the retention of such data.In addition to providing “opt in” and “opt out” options, the presentdisclosure contemplates providing notifications relating to the accessor use of personal information. For instance, a user may be notifiedupon downloading an app that their personal information data will beaccessed and then reminded again just before personal information datais accessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, communicationsessions can be established based on non-personal information data or abare minimum amount of personal information, such as the content beingrequested by the device associated with a user, other non-personalinformation available to the communication service providers, orpublicly available information.

What is claimed is:
 1. A first electronic device, comprising: an inputdevice; a display; one or more processors; and memory storing one ormore programs configured to be executed by the one or more processors,the one or more programs including instructions for: displaying, via thedisplay, a live video communication user interface that includes one ormore participant representations of one or more participants in a livevideo communication session; receiving, via the input device, a firstinput corresponding to a first media item; in response to receiving thefirst input, displaying, via the display, a first media representationof the first media item in the live video communication user interfaceconcurrently with the one or more participant representations, wherein afirst participant representation of the one or more participantrepresentations overlaps a portion of the first media representation;and while displaying the first media representation of the first mediaitem in the live video communication user interface concurrently withthe one or more participant representations: in response to adetermination that a second media item was selected by one of the one ormore participants in the live video communication session, concurrentlydisplaying in the live video communication user interface, via thedisplay, the first media representation of the first media item, asecond media representation of the second media item, and the one ormore participant representations, wherein a second participantrepresentation of the one or more participant representations overlaps aportion of one or more of the first media representation or the secondmedia representation.
 2. The electronic device of claim 1, wherein:receiving the first input corresponding to the first media item occurswhile displaying the live video communication user interface with theone or more participant representations.
 3. The electronic device ofclaim 1, the one or more programs further including instructions for: inresponse to receiving a second input corresponding to the first mediaitem: enlarging the first media representation of the first media itemin the live video communication user interface.
 4. The electronic deviceof claim 1, wherein the first participant representation of the one ormore participant representations includes a video live media stream, andwherein the second participant representation of the one or moreparticipant representations includes an audio-only live media stream. 5.The electronic device of claim 4, wherein the second participantrepresentation includes an avatar.
 6. The electronic device of claim 5,the one or more programs further including instructions for: in responseto receiving an audio input from a second participant associated withthe second participant representation, animating, via the display, theavatar of the second participant representation.
 7. The electronicdevice of claim 1, the one or more programs further includinginstructions for: adding a feature to the media representation of thefirst media item in the live video communication user interface.
 8. Anon-transitory computer-readable storage medium storing one or moreprograms configured to be executed by one or more processors of a firstelectronic device that is in communication with an input device and adisplay, the one or more programs including instructions for:displaying, via the display, a live video communication user interfacethat includes one or more participant representations of one or moreparticipants in a live video communication session; receiving, via theinput device, a first input corresponding to a first media item; inresponse to receiving the first input, displaying, via the display, afirst media representation of the first media item in the live videocommunication user interface concurrently with the one or moreparticipant representations, wherein a first participant representationof the one or more participant representations overlaps a portion of thefirst media representation; and while displaying the first mediarepresentation of the first media item in the live video communicationuser interface concurrently with the one or more participantrepresentations: in response to a determination that a second media itemwas selected by one of the one or more participants in the live videocommunication session, concurrently displaying in the live videocommunication user interface, via the display, the first mediarepresentation of the first media item, a second media representation ofthe second media item, and the one or more participant representations,wherein a second participant representation of the one or moreparticipant representations overlaps a portion of one or more of thefirst media representation or the second media representation.
 9. Thecomputer-readable storage medium of claim 8, wherein: receiving thefirst input corresponding to the first media item occurs whiledisplaying the live video communication user interface with the one ormore participant representations.
 10. The computer-readable storagemedium of claim 8, the one or more programs further includinginstructions for: in response to receiving a second input correspondingto the first media item: enlarging the first media representation of thefirst media item in the live video communication user interface.
 11. Thecomputer-readable storage medium of claim 8, wherein the firstparticipant representation of the one or more participantrepresentations includes a video live media stream, and wherein thesecond participant representation of the one or more participantrepresentations includes an audio-only live media stream.
 12. Thecomputer-readable storage medium of claim 11, wherein the secondparticipant representation includes an avatar.
 13. The computer-readablestorage medium of claim 12, the one or more programs further includinginstructions for: in response to receiving an audio input from a secondparticipant associated with the second participant representation,animating, via the display, the avatar of the second participantrepresentation.
 14. The computer-readable storage medium of claim 8, theone or more programs further including instructions for: adding afeature to the media representation of the first media item in the livevideo communication user interface.
 15. A method, comprising: at a firstelectronic device that is in communication with an input device and adisplay: displaying, via the display, a live video communication userinterface that includes one or more participant representations of oneor more participants in a live video communication session; receiving,via the input device, a first input corresponding to a first media item;in response to receiving the first input, displaying, via the display, afirst media representation of the first media item in the live videocommunication user interface concurrently with the one or moreparticipant representations, wherein a first participant representationof the one or more participant representations overlaps a portion of thefirst media representation; and while displaying the first mediarepresentation of the first media item in the live video communicationuser interface concurrently with the one or more participantrepresentations: in response to a determination that a second media itemwas selected by one of the one or more participants in the live videocommunication session, concurrently displaying in the live videocommunication user interface, via the display, the first mediarepresentation of the first media item, a second media representation ofthe second media item, and the one or more participant representations,wherein a second participant representation of the one or moreparticipant representations overlaps a portion of one or more of thefirst media representation or the second media representation.
 16. Themethod of claim 15, wherein: receiving the first input corresponding tothe first media item occurs while displaying the live videocommunication user interface with the one or more participantrepresentations.
 17. The method of claim 15, further comprising: inresponse to receiving a second input corresponding to the first mediaitem: enlarging the first media representation of the first media itemin the live video communication user interface.
 18. The method of claim15, wherein the first participant representation of the one or moreparticipant representations includes a video live media stream, andwherein the second participant representation of the one or moreparticipant representations includes an audio-only live media stream.19. The method of claim 18, wherein the second participantrepresentation includes an avatar.
 20. The method of claim 19, furthercomprising: in response to receiving an audio input from a secondparticipant associated with the second participant representation,animating, via the display, the avatar of the second participantrepresentation.
 21. The method of claim 15, further comprising: adding afeature to the media representation of the first media item in the livevideo communication user interface.